An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU

Abstract

Objectives: Sepsis is among the leading causes of morbidity, mortality, and cost overruns in critically ill patients. Early intervention with antibiotics improves survival in septic patients. However, no clinically validated system exists for real-time prediction of sepsis onset. We aimed to develop and validate an Artificial Intelligence Sepsis Expert algorithm for early prediction of sepsis.

Design: Observational cohort study.

Setting: Academic medical center from January 2013 to December 2015.

Patients: Over 31,000 admissions to the ICUs at two Emory University hospitals (development cohort), in addition to over 52,000 ICU patients from the publicly available Medical Information Mart for Intensive Care-III ICU database (validation cohort). Patients who met the Third International Consensus Definitions for Sepsis (Sepsis-3) prior to or within 4 hours of their ICU admission were excluded, resulting in roughly 27,000 and 42,000 patients within our development and validation cohorts, respectively.

Interventions: None.

Measurements and main results: High-resolution vital signs time series and electronic medical record data were extracted. A set of 65 features (variables) were calculated on hourly basis and passed to the Artificial Intelligence Sepsis Expert algorithm to predict onset of sepsis in the proceeding T hours (where T = 12, 8, 6, or 4). Artificial Intelligence Sepsis Expert was used to predict onset of sepsis in the proceeding T hours and to produce a list of the most significant contributing factors. For the 12-, 8-, 6-, and 4-hour ahead prediction of sepsis, Artificial Intelligence Sepsis Expert achieved area under the receiver operating characteristic in the range of 0.83-0.85. Performance of the Artificial Intelligence Sepsis Expert on the development and validation cohorts was indistinguishable.

Conclusions: Using data available in the ICU in real-time, Artificial Intelligence Sepsis Expert can accurately predict the onset of sepsis in an ICU patient 4-12 hours prior to clinical recognition. A prospective study is necessary to determine the clinical utility of the proposed sepsis prediction model.

FIGURE 1

Receiver Operating Characteristic (ROC) curves for predicting t_sepsis 4 hours in advance. Catching 85% of the septic patients yielded 30% false alarms (SP=0.70) within the training set (left panel) and 33% false alarms (SP=0.67) within the testing set (right panel). See Table 2 for information on the false alarms.

FIGURE 2

Summary of training set (dashed lines) and testing set (solid lines) prediction performance of AISE on the Emory cohort. Area under the ROC curve (AUROC) as a function of prediction window shows a decreasing pattern. Across all windows, the best performance is achieved for predicting t_SOFA, followed by t_sepsis, and finally t_onset. A close agreement between the training set and testing set performance indicates good generalizability.

FIGURE 3

An illustrative example of the prediction performance of AISE. Hourly calculated Sequential Organ Failure Assessment (SOFA) Score, Sepsis-3 definition, and the AISE score are shown for one patient in Panel (A). Superimposed on the figure is the order-time of three blood cultures, and the administration-time of two antibiotics. In Panel (B), commonly recorded hourly vital signs of the patient, including heart rate (HR), Mean Arterial Blood Pressure (MAP), Respiratory Rate (RESP), Temperature (TEMP), Oxygen Saturation (O₂Sat) and the Glascow Coma Score (GCS) are shown. Panel (C) shows the most significant features contributing to the AISE score (for clarity of presentation only selected time-points are shown). Notably, around 4pm on December 20^th, roughly 8 hours prior to any change in the SOFA score, the AISE score starts to increase. The top contributing factors were slight changes in HR, RESP, and TEMP, given that the patient had surgery in the past 12 hours with a contaminated wound, and was on a mechanical ventilator. Close to midnight on December 21^st, other factors such as multiscale entropy of MAP time series (BPV1), GCS, and Lactate show abnormal changes. Five hours later, the patient met the sepsis-3 definition of sepsis.