Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep 23:3:e60020.
doi: 10.2196/60020.

Impact of a Digital Scribe System on Clinical Documentation Time and Quality: Usability Study

Marieke Meija van Buchem  1 , Ilse M J Kant  2 , Liza King  3 , Jacqueline Kazmaier  3 , Ewout W Steyerberg  4 , Martijn P Bauer  5
Affiliations

Impact of a Digital Scribe System on Clinical Documentation Time and Quality: Usability Study

Marieke Meija van Buchem et al. JMIR AI. .

Abstract

Background: Physicians spend approximately half of their time on administrative tasks, which is one of the leading causes of physician burnout and decreased work satisfaction. The implementation of natural language processing-assisted clinical documentation tools may provide a solution.

Objective: This study investigates the impact of a commercially available Dutch digital scribe system on clinical documentation efficiency and quality.

Methods: Medical students with experience in clinical practice and documentation (n=22) created a total of 430 summaries of mock consultations and recorded the time they spent on this task. The consultations were summarized using 3 methods: manual summaries, fully automated summaries, and automated summaries with manual editing. We then randomly reassigned the summaries and evaluated their quality using a modified version of the Physician Documentation Quality Instrument (PDQI-9). We compared the differences between the 3 methods in descriptive statistics, quantitative text metrics (word count and lexical diversity), the PDQI-9, Recall-Oriented Understudy for Gisting Evaluation scores, and BERTScore.

Results: The median time for manual summarization was 202 seconds against 186 seconds for editing an automatic summary. Without editing, the automatic summaries attained a poorer PDQI-9 score than manual summaries (median PDQI-9 score 25 vs 31, P<.001, ANOVA test). Automatic summaries were found to have higher word counts but lower lexical diversity than manual summaries (P<.001, independent t test). The study revealed variable impacts on PDQI-9 scores and summarization time across individuals. Generally, students viewed the digital scribe system as a potentially useful tool, noting its ease of use and time-saving potential, though some criticized the summaries for their greater length and rigid structure.

Conclusions: This study highlights the potential of digital scribes in improving clinical documentation processes by offering a first summary draft for physicians to edit, thereby reducing documentation time without compromising the quality of patient records. Furthermore, digital scribes may be more beneficial to some physicians than to others and could play a role in improving the reusability of clinical documentation. Future studies should focus on the impact and quality of such a system when used by physicians in clinical practice.

Keywords: AI; LLM; LLMs; ML; NLP; algorithm; algorithms; analytics; artificial intelligence; automate; automation; clinical documentation; deep learning; documentation; documentation quality; documentation time; implementation; large language model; large language models; machine learning; model; models; natural language processing; pilot studies; pilot study; practical model; practical models.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: JK, LK, and MB are employees of Autoscriber. Their affiliation with Autoscriber did not influence the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The other authors, who are not affiliated with Autoscriber, contributed independently to this work, ensuring unbiased data interpretation and conclusions.

Figures

Figure 1
Figure 1
Flowchart showing the 3 different summarization methods and consecutive evaluation.
See this image and copyright information in PMC

References

    1. Shanafelt TD, West CP, Sinsky C, Trockel M, Tutty M, Satele DV, Carlasare LE, Dyrbye LN. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017. Mayo Clin Proc. 2019;94(9):1681–1694. doi: 10.1016/j.mayocp.2018.10.023. https://linkinghub.elsevier.com/retrieve/pii/S0025-6196(18)30938-8 S0025-6196(18)30938-8 - DOI - PubMed
    1. Taking Action Against Clinician Burnout: A Systems Approach to Professional Well-Being. Washington DC: The National Academies Press; 2019. - PubMed
    1. Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, Sinsky CA, Gilchrist VJ. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med. 2017;15(5):419–426. doi: 10.1370/afm.2121. http://www.annfammed.org/cgi/pmidlookup?view=long&pmid=28893811 15/5/419 - DOI - PMC - PubMed
    1. Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, Blike G. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016;165(11):753–760. doi: 10.7326/M16-0961.2546704 - DOI - PubMed
    1. Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, Wang W, Luft HS. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017;36(4):655–662. doi: 10.1377/hlthaff.2016.0811. https://europepmc.org/abstract/MED/28373331 36/4/655 - DOI - PMC - PubMed

LinkOut - more resources