Challenges to the Reproducibility of Machine Learning Models in Health Care

Andrew L Beam^{1

2}, Arjun K Manrai^{2

3}, Marzyeh Ghassemi^{4

5}

Affiliations

¹ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
² Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts.
³ Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts.
⁴ Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
⁵ Vector Institute, Toronto, Ontario, Canada.

Andrew L Beam et al. JAMA. 2020.

. 2020 Jan 28;323(4):305-306.

doi: 10.1001/jama.2019.20866.

Andrew L Beam^{1

2}, Arjun K Manrai^{2

3}, Marzyeh Ghassemi^{4

5}

¹ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
² Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts.
³ Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts.
⁴ Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
⁵ Vector Institute, Toronto, Ontario, Canada.

No abstract available

Conflict of interest statement

Conflict of Interest Disclosures: None reported.

1. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi: 10.1371/journal.pmed.0020124 - DOI - PMC - PubMed
1. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716 - DOI - PubMed
1. Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–1318. doi:10.1001/jama.2017.18391 - DOI - PubMed
1. Hinton G Deep learning—a technology with the potential to transform health care. JAMA. 2018;320 (11):1101–1102. doi:10.1001/jama.2018.11100 - DOI - PubMed
1. Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D. Deep reinforcement learning that matters. Presented at: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18); February 2–7, 2018; New Orleans, Louisiana.