Contextual Embeddings from Clinical Notes Improves Prediction of Sepsis

Abstract

Sepsis, a life-threatening organ dysfunction, is a clinical syndrome triggered by acute infection and affects over 1 million Americans every year. Untreated sepsis can progress to septic shock and organ failure, making sepsis one of the leading causes of morbidity and mortality in hospitals. Early detection of sepsis and timely antibiotics administration is known to save lives. In this work, we design a sepsis prediction algorithm based on data from electronic health records (EHR) using a deep learning approach. While most existing EHR-based sepsis prediction models utilize structured data including vitals, labs, and clinical information, we show that incorporation of features based on clinical texts, using a pre-trained neural language representation model, allows for incorporation of unstructured data without an explicit need for ontology-based named-entity recognition and classification. The proposed model is trained on a large critical care database of over 40,000 patients, including 2805 septic patients, and is compared against competing baseline models. In comparison to a baseline model based on structured data alone, incorporation of clinical texts improved AUC from 0.81 to 0.84. Our findings indicate that incorporation of clinical text features via a pre-trained language representation model can improve early prediction of sepsis and reduce false alarms.

Figure 1:

Schematic diagram of the proposed model, including preprocessing pipeline and predictive model architec- ture. The preprocessing pipeline includes retrieving the contextual embedding of each of the clinical notes (on hourly basis) by averaging the ClinicalBERT embedding representation of sentences within each document. The resulting representations are then concatenated with the structured clinical data (vitals and laboratory values) and fed into an LSTM model for early prediction of sepsis.

Figure 2:

Receiver Operating Characteristic (ROC) Curves for all four models. ClinicalBERT embeddings of notes alone (Model I) is our baseline method reached the Area Under the ROC Curve (AUC) of 0.74. Structured clinical data from vital signs and laboratory measurements (Model II) achieved an AUC performance of 0.81. combining TF-IDF features with the clinical data (Model III) achieved an AUC of 0.82. Combining both structural clinical data with ClinicalBERT embeddings (Model IV) achieved the best AUC performance.