Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep 1;183(9):1028-1030.
doi: 10.1001/jamainternmed.2023.2909.

Chatbot vs Medical Student Performance on Free-Response Clinical Reasoning Examinations

Eric Strong  1 , Alicia DiGiammarino  2 , Yingjie Weng  3 , Andre Kumar  1 , Poonam Hosamani  1 , Jason Hom  1 , Jonathan H Chen  1   4   5
Affiliations

Chatbot vs Medical Student Performance on Free-Response Clinical Reasoning Examinations

Eric Strong et al. JAMA Intern Med. .
No abstract available

Plain language summary

This study compares performance on free-response clinical reasoning examinations of first- and second-year medical students vs 2 models of a popular chatbot.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Hom reported receiving personal fees from More Health, Inc outside the submitted work. Dr Chen reported receiving grants from Stanford Artificial Intelligence in Medicine and Imaging–Human-Centered Artificial Intelligence Partnership Grant, Google Inc, the Doris Duke Foundation COVID-19 Fund to Retain Clinical Scientists, the National Institute on Drug Abuse, National Institutes of Health Clinical Trials Network, and the American Heart Association Strategically Focused Research Network–Diversity in Clinical Trials during the conduct of the study and being co-owner of Reaction Explorer LLC and receiving expert witness fees from Younker Hyde MacFarlance PLLC and Sutton Pierce outside the submitted work. No other disclosures were reported.

Update of

References

    1. Open AI. ChatGPT: optimizing language models for dialogue. OpenAI. Published November 30, 2022. Accessed February 1, 2023. https://openai.com/blog/chatgpt/
    1. Nori H, King N, McKinney SM, et al. . Capabilities of GPT-4 on medical challenge problems. arXiv. Preprint posted online March 20, 2023. doi:10.48550/arXiv.2303.13375 - DOI
    1. Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023;388(13):1233-1239. doi:10.1056/NEJMsr2214184 - DOI - PubMed
    1. American Board of Internal Medicine. Doctors maintaining ABIM board certification will soon be able to access an electronic resource they use in practice during periodic knowledge assessments. September 27, 2017. Accessed February 2, 2023. https://www.abim.org/media-center/press-releases/abim-open-book-assessme....