Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 30;8(4):ooaf082.
doi: 10.1093/jamiaopen/ooaf082. eCollection 2025 Aug.

Comparing artificial intelligence- vs clinician-authored summaries of simulated primary care electronic health records

Lara Shemtob  1 , Abdullah Nouri  2 , Adam Harvey-Sullivan  3 , Connor S Qiu  1 , Jonathan Martin  4 , Martha Martin  1 , Sara Noden  5 , Tanveer Rob  6 , Ana L Neves  1 , Azeem Majeed  1 , Jonathan Clarke  7 , Thomas Beaney  8
Affiliations

Comparing artificial intelligence- vs clinician-authored summaries of simulated primary care electronic health records

Lara Shemtob et al. JAMIA Open. .

Abstract

Objective: To compare clinical summaries generated from simulated patient primary care electronic health records (EHRs) by GPT-4, to summaries generated by clinicians on multiple domains of quality including utility, concision, accuracy, and bias.

Materials and methods: Seven primary care physicians generated 70 simulated patient EHR notes, each representing 10 patient contacts with the practice over at least 2 years. Each record was summarized by a different clinician and by GPT-4. artificial intelligence (AI)- and clinician-authored summaries were rated blind by clinicians according to 8 domains of quality and an overall rating.

Results: The median time taken for a clinician to read through and assimilate the information in the EHRs before summarizing, was 7 minutes. Clinicians rated clinician-authored summaries higher than AI-authored summaries overall (7.39 vs 7.00 out of 10; P = .02), but with greater variability in clinician-authored summary ratings. AI and clinician-authored summaries had similar accuracy and AI-authored summaries were less likely to omit important information and more likely to use patient-friendly language.

Discussion: Although AI-authored summaries were rated slightly lower overall compared with clinician-authored summaries, they demonstrated similar accuracy and greater consistency. This demonstrates potential applications for generating summaries in primary care, particularly given the substantial time taken for clinicians to undertake this work.

Conclusion: The results suggest the feasibility, utility and acceptability of using AI-authored summaries to integrate into EHRs to support clinicians in primary care. AI summarization tools have the potential to improve healthcare productivity, including by enabling clinicians to spend more time on direct patient care.

Keywords: electronic health records; generative AI; health informatics; large language models; primary care.

PubMed Disclaimer

Conflict of interest statement

The authors have no competing interests to declare.

Figures

Figure 1.
Figure 1.
Schematic of study design.
Figure 2.
Figure 2.
Example template and first (of 10) free text entries for a simulated patient record.
Figure 3.
Figure 3.
Qualities of summaries rated by clinicians on 5-point Likert scale.
Figure 4.
Figure 4.
Pairwise distances between the 3 ratings assigned by clinicians for clinician and AI-authored summaries.
Figure 5.
Figure 5.
Combined figure of all responses (cross-hatched bars are clinician-authored, plain are AI-authored).
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Steinkamp J, Kantrowitz JJ, Airan-Javia S. Prevalence and sources of duplicate information in the electronic medical record. JAMA Netw Open. 2022;5:e2233348. - PMC - PubMed
    1. Millares Martin P. Consultation analysis: use of free text versus coded text. Health Technol. 2021;11:349-357. - PMC - PubMed
    1. Pinevich Y, Clark KJ, Harrison AM, Pickering BW, Herasevich V. Interaction time with electronic health records: a systematic review. Appl Clin Inform. 2021;12:788-799. - PMC - PubMed
    1. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and Time-Motion observations. Ann Fam Med. 2017;15:419-426. - PMC - PubMed
    1. Nijor S, Rallis G, Lad N, Gokcen E. Patient safety issues from information overload in electronic medical records. J Patient Saf. 2022;18:e999-e1003. - PMC - PubMed

LinkOut - more resources