A method to automate the discharge summary hospital course for neurology patients

Abstract

Objective: Generation of automated clinical notes has been posited as a strategy to mitigate physician burnout. In particular, an automated narrative summary of a patient's hospital stay could supplement the hospital course section of the discharge summary that inpatient physicians document in electronic health record (EHR) systems. In the current study, we developed and evaluated an automated method for summarizing the hospital course section using encoder-decoder sequence-to-sequence transformer models.

Materials and methods: We fine-tuned BERT and BART models and optimized for factuality through constraining beam search, which we trained and tested using EHR data from patients admitted to the neurology unit of an academic medical center.

Results: The approach demonstrated good ROUGE scores with an R-2 of 13.76. In a blind evaluation, 2 board-certified physicians rated 62% of the automated summaries as meeting the standard of care, which suggests the method may be useful clinically.

Discussion and conclusion: To our knowledge, this study is among the first to demonstrate an automated method for generating a discharge summary hospital course that approaches a quality level of what a physician would write.

Keywords: abstractive summarization; automated clinical notes; automated patient summary; clinician burnout; machine learning; natural language processing.

Figure 1.

Data flow that shows how EHR data is segmented into 3 separate sections through the following transformer models referred to as the “day-to-day approach”: (1) HPI summarization, (2) daily narrative document classification and summarization, and (3) follow-up extraction classification. The automated summary is constructed by chronologically assembling the results.

Figure 2.

A motivational example of how clinical summaries can hallucinate. Inconsistent medical terms are highlighted in red font. In this example, the proposed BART model with constrained beam search for medical terminology removes the clinical inconsistencies from the HPI summary.

Figure 3.

The dependency arc entailment (DAE) model was pretrained on BERT XSum. If the arc is nonfactual, then the sentence summary is marked as nonfactual.

Figure 4.

Box plot for the Likert scores for quality, readability, factuality, and completeness of the automated and physician-written summaries by the physician reviewers.