Challenges to the Reproducibility of Machine Learning Models in Health Care

doi:10.1001/jama.2019.20866

. 2020 Jan 28;323(4):305-306.

doi: 10.1001/jama.2019.20866.

Challenges to the Reproducibility of Machine Learning Models in Health Care

Andrew L Beam^{1

2}, Arjun K Manrai^{2

3}, Marzyeh Ghassemi^{4

5}

Affiliations

¹ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
² Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts.
³ Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts.
⁴ Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
⁵ Vector Institute, Toronto, Ontario, Canada.

PMID: 31904799
PMCID: PMC7335677
DOI: 10.1001/jama.2019.20866

Challenges to the Reproducibility of Machine Learning Models in Health Care

Andrew L Beam et al. JAMA. 2020.

. 2020 Jan 28;323(4):305-306.

doi: 10.1001/jama.2019.20866.

Authors

Andrew L Beam^{1

2}, Arjun K Manrai^{2

3}, Marzyeh Ghassemi^{4

5}

Affiliations

¹ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
² Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts.
³ Computational Health Informatics Program, Boston Children's Hospital, Boston, Massachusetts.
⁴ Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.
⁵ Vector Institute, Toronto, Ontario, Canada.

PMID: 31904799
PMCID: PMC7335677
DOI: 10.1001/jama.2019.20866

No abstract available

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: None reported.

Cited by

Training Strategies for Radiology Deep Learning Models in Data-limited Scenarios.
Candemir S, Nguyen XV, Folio LR, Prevedello LM. Candemir S, et al. Radiol Artif Intell. 2021 Oct 6;3(6):e210014. doi: 10.1148/ryai.2021210014. eCollection 2021 Nov. Radiol Artif Intell. 2021. PMID: 34870217 Free PMC article. Review.
Challenges and Future Prospects of Precision Medicine in Psychiatry.
Manchia M, Pisanu C, Squassina A, Carpiniello B. Manchia M, et al. Pharmgenomics Pers Med. 2020 Apr 23;13:127-140. doi: 10.2147/PGPM.S198225. eCollection 2020. Pharmgenomics Pers Med. 2020. PMID: 32425581 Free PMC article. Review.
AI-Based Metamaterial Design.
Tezsezen E, Yigci D, Ahmadpour A, Tasoglu S. Tezsezen E, et al. ACS Appl Mater Interfaces. 2024 Jun 12;16(23):29547-29569. doi: 10.1021/acsami.4c04486. Epub 2024 May 29. ACS Appl Mater Interfaces. 2024. PMID: 38808674 Free PMC article. Review.
Machine Learning Applications in the Diagnosis of Benign and Malignant Hematological Diseases.
Muhsen IN, Shyr D, Sung AD, Hashmi SK. Muhsen IN, et al. Clin Hematol Int. 2020 Dec 21;3(1):13-20. doi: 10.2991/chi.k.201130.001. eCollection 2021 Mar. Clin Hematol Int. 2020. PMID: 34595462 Free PMC article. Review.
Molecular de-extinction of ancient antimicrobial peptides enabled by machine learning.
Maasch JRMA, Torres MDT, Melo MCR, de la Fuente-Nunez C. Maasch JRMA, et al. Cell Host Microbe. 2023 Aug 9;31(8):1260-1274.e6. doi: 10.1016/j.chom.2023.07.001. Epub 2023 Jul 28. Cell Host Microbe. 2023. PMID: 37516110 Free PMC article.

See all "Cited by" articles

References

1. Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi: 10.1371/journal.pmed.0020124 - DOI - PMC - PubMed
1. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716 - DOI - PubMed
1. Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–1318. doi:10.1001/jama.2017.18391 - DOI - PubMed
1. Hinton G Deep learning—a technology with the potential to transform health care. JAMA. 2018;320 (11):1101–1102. doi:10.1001/jama.2018.11100 - DOI - PubMed
1. Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D. Deep reinforcement learning that matters. Presented at: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18); February 2–7, 2018; New Orleans, Louisiana.

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- The Lens - Patent Citations Database

[1] Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi: 10.1371/journal.pmed.0020124 - DOI - PMC - PubMed

[2] Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi: 10.1371/journal.pmed.0020124 - DOI - PMC - PubMed

[3] Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716 - DOI - PubMed

[4] Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):aac4716. doi:10.1126/science.aac4716 - DOI - PubMed

[5] Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–1318. doi:10.1001/jama.2017.18391 - DOI - PubMed

[6] Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):1317–1318. doi:10.1001/jama.2017.18391 - DOI - PubMed

[7] Hinton G Deep learning—a technology with the potential to transform health care. JAMA. 2018;320 (11):1101–1102. doi:10.1001/jama.2018.11100 - DOI - PubMed

[8] Hinton G Deep learning—a technology with the potential to transform health care. JAMA. 2018;320 (11):1101–1102. doi:10.1001/jama.2018.11100 - DOI - PubMed

[9] Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D. Deep reinforcement learning that matters. Presented at: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18); February 2–7, 2018; New Orleans, Louisiana.

[10] Henderson P, Islam R, Bachman P, Pineau J, Precup D, Meger D. Deep reinforcement learning that matters. Presented at: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18); February 2–7, 2018; New Orleans, Louisiana.

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Challenges to the Reproducibility of Machine Learning Models in Health Care

Affiliations

Challenges to the Reproducibility of Machine Learning Models in Health Care

Authors

Affiliations

Conflict of interest statement

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources