Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep 1;183(9):1026-1027.
doi: 10.1001/jamainternmed.2023.2561.

Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine Residents

Ashwin Nayak  1 , Matthew S Alkaitis  1 , Kristen Nayak  1 , Margaret Nikolov  2 , Kevin P Weinfurt  3 , Kevin Schulman  1   2   4
Affiliations

Comparison of History of Present Illness Summaries Generated by a Chatbot and Senior Internal Medicine Residents

Ashwin Nayak et al. JAMA Intern Med. .
No abstract available

Plain language summary

This prognostic study assesses the ability of a chatbot to write a history of present illness compared with senior internal medicine residents.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: None reported.

Figures

Figure.
Figure.. Prompt Engineering With the Chatbot
A total of 10 HPIs per script (30 total) were generated in each round of prompt engineering and assessed for major errors such as reported information in the HPIs that was not present in the source dialogue (hallucinations), the presence of atypical terminology, and structural issues. The HPIs without any major errors were considered acceptable. Prompts were refined in each round to minimize error and maximize acceptance. HPI indicates history of present illness.
See this image and copyright information in PMC

Comment in

References

    1. Online ChatGPT: optimizing language models for dialogue. Published February 10, 2023. Accessed February 11, 2023. https://online-chatgpt.com/
    1. van Dis EAM, Bollen J, Zuidema W, van Rooij R, Bockting CL. ChatGPT: five priorities for research. Nature. 2023;614(7947):224-226. doi:10.1038/d41586-023-00288-7 - DOI - PubMed
    1. Gaffney A, Woolhandler S, Cai C, et al. . Medical Documentation Burden Among US Office-Based Physicians in 2019: A National Study. JAMA Intern Med. 2022;182(5):564-566. doi:10.1001/jamainternmed.2022.0372 - DOI - PMC - PubMed
    1. Schulman KA, Berlin JA, Harless W, et al. . The effect of race and sex on physicians’ recommendations for cardiac catheterization. N Engl J Med. 1999;340(8):618-626. doi:10.1056/NEJM199902253400806 - DOI - PubMed
    1. Bang Y, Cahyawijaya S, Lee N, et al. . A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv. Preprint posted online February 8, 2023. doi:10.48550/arXiv.2302.04023 - DOI