. 2023 Sep 1;177(9):977-979.

doi: 10.1001/jamapediatrics.2023.2373.

Performance of a Large Language Model on Practice Questions for the Neonatal Board Examination

Kristyn Beam¹, Puneet Sharma², Bhawesh Kumar³, Cindy Wang⁴, Dara Brodsky¹, Camilia R Martin⁵, Andrew Beam⁶

Affiliations

¹ Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, Massachusetts.
² Division of Newborn Medicine, Boston Children's Hospital, Boston, Massachusetts.
³ Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
⁴ Department of Statistics, Harvard University, Cambridge, Massachusetts.
⁵ Division of Neonatology, Weill Cornell Medicine, New York, New York.
⁶ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.

PMID: 37459084
PMCID: PMC10352922
DOI: 10.1001/jamapediatrics.2023.2373

Performance of a Large Language Model on Practice Questions for the Neonatal Board Examination

Kristyn Beam et al. JAMA Pediatr. 2023.

. 2023 Sep 1;177(9):977-979.

doi: 10.1001/jamapediatrics.2023.2373.

Authors

Kristyn Beam¹, Puneet Sharma², Bhawesh Kumar³, Cindy Wang⁴, Dara Brodsky¹, Camilia R Martin⁵, Andrew Beam⁶

Affiliations

¹ Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, Massachusetts.
² Division of Newborn Medicine, Boston Children's Hospital, Boston, Massachusetts.
³ Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
⁴ Department of Statistics, Harvard University, Cambridge, Massachusetts.
⁵ Division of Neonatology, Weill Cornell Medicine, New York, New York.
⁶ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.

PMID: 37459084
PMCID: PMC10352922
DOI: 10.1001/jamapediatrics.2023.2373

No abstract available

Plain language summary

This Diagnostic/Prognostic Study evaluates the performance of a large language model in generating answers to practice questions for the neonatal-perinatal board examination.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: None reported.

Cited by

Can ChatGPT pass the MRCP (UK) written examinations? Analysis of performance and errors using a clinical decision-reasoning framework.
Maitland A, Fowkes R, Maitland S. Maitland A, et al. BMJ Open. 2024 Mar 15;14(3):e080558. doi: 10.1136/bmjopen-2023-080558. BMJ Open. 2024. PMID: 38490655 Free PMC article.
Assessing the adherence of large language models to clinical practice guidelines in Chinese medicine: a content analysis.
Zhao W, Lai H, Pan B, Huang J, Xia D, Bai C, Liu J, Liu J, Jin Y, Shang H, Liu J, Shi N, Liu J, Chen Y, Estill J, Ge L. Zhao W, et al. Front Pharmacol. 2025 Jul 25;16:1649041. doi: 10.3389/fphar.2025.1649041. eCollection 2025. Front Pharmacol. 2025. PMID: 40786055 Free PMC article.
Performance of Publicly Available Large Language Models on Internal Medicine Board-style Questions.
Tarabanis C, Zahid S, Mamalis M, Zhang K, Kalampokis E, Jankelson L. Tarabanis C, et al. PLOS Digit Health. 2024 Sep 17;3(9):e0000604. doi: 10.1371/journal.pdig.0000604. eCollection 2024 Sep. PLOS Digit Health. 2024. PMID: 39288137 Free PMC article.
ChatGPT's performance in German OB/GYN exams - paving the way for AI-enhanced medical education and clinical practice.
Riedel M, Kaefinger K, Stuehrenberg A, Ritter V, Amann N, Graf A, Recker F, Klein E, Kiechle M, Riedel F, Meyer B. Riedel M, et al. Front Med (Lausanne). 2023 Dec 13;10:1296615. doi: 10.3389/fmed.2023.1296615. eCollection 2023. Front Med (Lausanne). 2023. PMID: 38155661 Free PMC article.
Assessment of the clinical knowledge of ChatGPT-4 in neonatal-perinatal medicine: a comparative analysis with ChatGPT-3.5.
Sharma P, Luo G, Wang C, Brodsky D, Martin CR, Beam A, Beam K. Sharma P, et al. J Perinatol. 2024 Sep;44(9):1365-1366. doi: 10.1038/s41372-024-01912-8. Epub 2024 Feb 24. J Perinatol. 2024. PMID: 38402349 Free PMC article. No abstract available.

See all "Cited by" articles

References

1. Schulman J, Zoph B, Kim C, et al. . ChatGPT: optimizing language models for dialogue. OpenAI . Published November 30, 2022. Accessed February 28, 2023. https://chat.openai.com
1. Levine DM, Tuwani R, Kompa B, et al. . The diagnostic and triage accuracy of the gpt-3 artificial intelligence model. medRxiv. Published online February 1, 2023. doi:10.1101/2023.01.30.23285067 - DOI - PubMed
1. Kung TH, Cheatham M, Medinilla A, et al. . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. doi:10.1371/journal.pdig.0000198 - DOI - PMC - PubMed
1. Morton S, Ehret D, Ghanta S, Sajti E, Walsh B. In: Brodsky D, Martin CR, eds. Neonatology Review: Q & A. 3rd ed. Lulu; 2015.
1. Singhal K, Azizi S, Tu T, et al. . Large language models encode clinical knowledge. arXiv. Published online December 26, 2022. https://arxiv.org/abs/2212.13138

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

T32 HD098061/HD/NICHD NIH HHS/United States

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Performance of a Large Language Model on Practice Questions for the Neonatal Board Examination

Affiliations

Performance of a Large Language Model on Practice Questions for the Neonatal Board Examination

Authors

Affiliations

Plain language summary

Conflict of interest statement

Similar articles

Cited by

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources