Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data

Abstract

Sepsis is a leading cause of mortality and early identification improves survival. With increasing digitalization of health care data automated sepsis prediction models hold promise to aid in prompt recognition. Most previous studies have focused on the intensive care unit (ICU) setting. Yet only a small proportion of sepsis develops in the ICU and there is an apparent clinical benefit to identify patients earlier in the disease trajectory. In this cohort of 82,852 hospital admissions and 8038 sepsis episodes classified according to the Sepsis-3 criteria, we demonstrate that a machine learned score can predict sepsis onset within 48 h using sparse routine electronic health record data outside the ICU. Our score was based on a causal probabilistic network model-SepsisFinder-which has similarities with clinical reasoning. A prediction was generated hourly on all admissions, providing a new variable was registered. Compared to the National Early Warning Score (NEWS2), which is an established method to identify sepsis, the SepsisFinder triggered earlier and had a higher area under receiver operating characteristic curve (AUROC) (0.950 vs. 0.872), as well as area under precision-recall curve (APR) (0.189 vs. 0.149). A machine learning comparator based on a gradient-boosting decision tree model had similar AUROC (0.949) and higher APR (0.239) than SepsisFinder but triggered later than both NEWS2 and SepsisFinder. The precision of SepsisFinder increased if screening was restricted to the earlier admission period and in episodes with bloodstream infection. Furthermore, the SepsisFinder signaled median 5.5 h prior to antibiotic administration. Identifying a high-risk population with this method could be used to tailor clinical interventions and improve patient care.

Figure 1

The concept of the sepsis prediction algorithm. The black line represents the SepsisFinder model with predictions marked by black dots. The red dot represents positive alarms. The grey line and dots illustrate silencing 48 h after positive alarms. The alarm threshold is illustrated by the dotted line. The red shaded area represents the time window for considering true positive alarms or false negative predictions. All predictions crossing the alarm threshold outside of the red shaded area were considered false positives. Predictions occurring below the alarm threshold and outside of the red shaded area represent true negative predictions. The upper panel represents a hospital episode with sepsis, but without any positive predictions and only a false negative prediction in the red shaded area. The middle panel shows an episode without sepsis, but with one false positive prediction and several true negative predictions. The lower panel represents a hospital episode with sepsis where both one false positive alarm and one true positive alarm were registered, with model explanations shown below the risk score trace at two selected points: the lowest score in the episode and the first alarm. The explanation plots show the Bayes Factor contributions of the evidence available at the respective times. In the model explanation plots, the blue bars show the degree to which a measurement increases the risk score, while the red bars show the degree to which the risk score is reduced. The Bayes Factor is defined as the ratio of the posterior and prior odds ratios e.g. $B=\frac{P(x|\epsilon \prime )/P(y|\epsilon \prime )}{P(x)/P(y)}$ where $x$ is the hypothesis (e.g. sepsis), $y$ is the alternative hypothesis (e.g. no sepsis) and $\epsilon \prime$ is the evidence, or a subset thereof. Respiratory rate (RR), heart rate (HR), mean arterial pressure (MAP), c-reactive protein (CRP).

Figure 2

The discriminative performance of the algorithms in the validation set. The left panel shows a receiver operating characteristic curve, and the right panel shows a precision recall curve for the prediction of sepsis within 48 h using SepsisFinder (blue line), the NEWS2 (green line) and the GBDT model (yellow line). Operating alarm thresholds corresponding to NEWS2 equal to 5 and 7 points have been marked for both scores. For SepsisFinder and GBDT, an additional alarm threshold corresponding to approximately 85% sensitivity has been marked. The blue shaded area illustrates the suggested clinically applicable region, and the grey shaded area illustrates the suggested clinically inapplicable region (specificity < 90% and precision < 15%) of model performance. SepsisFinder (SF), Area Under Receiver Operating Characteristic curve (AUC), Area Under Precision Recall curve (APR), National Early Warning Score 2 (NEWS2), and gradient-boosting decision tree (GBDT).

Figure 3

Performance of SepsisFinder in episodes where a sepsis event occurred based on fixed time points 24 h before sepsis onset for three operationalized alarm thresholds. The alarm thresholds were chosen based on sensitivity (recall) matched to NEWS2 equal to 5 points (sensitivity 20%) and 7 points (sensitivity 42%) as well as sensitivity 85%. Since sepsis occurred at all times from admission to discharge, and predictions were only based on data from the current hospital episode, a dotted line has been added to represents the detectable limit for sepsis onset. National Early Warning Score 2 (NEWS2), Recall (Rec), Hours (h), Precision/positive predictive value (Prec).