Diagnostic and prognostic capabilities of a biomarker and EMR-based machine learning algorithm for sepsis

Abstract

Sepsis is a major cause of mortality among hospitalized patients worldwide. Shorter time to administration of broad-spectrum antibiotics is associated with improved outcomes, but early recognition of sepsis remains a major challenge. In a two-center cohort study with prospective sample collection from 1400 adult patients in emergency departments suspected of sepsis, we sought to determine the diagnostic and prognostic capabilities of a machine-learning algorithm based on clinical data and a set of uncommonly measured biomarkers. Specifically, we demonstrate that a machine-learning model developed using this dataset outputs a score with not only diagnostic capability but also prognostic power with respect to hospital length of stay (LOS), 30-day mortality, and 3-day inpatient re-admission both in our entire testing cohort and various subpopulations. The area under the receiver operating curve (AUROC) for diagnosis of sepsis was 0.83. Predicted risk scores for patients with septic shock were higher compared with patients with sepsis but without shock (p < 0.0001). Scores for patients with infection and organ dysfunction were higher compared with those without either condition (p < 0.0001). Stratification based on predicted scores of the patients into low, medium, and high-risk groups showed significant differences in LOS (p < 0.0001), 30-day mortality (p < 0.0001), and 30-day inpatient readmission (p < 0.0001). In conclusion, a machine-learning algorithm based on electronic medical record (EMR) data and three nonroutinely measured biomarkers demonstrated good diagnostic and prognostic capability at the time of initial blood culture.

FIGURE 1

ROC curves of the algorithm in the testing cohort for (a) all patients, (b) SOFA‐positive patients, and (c) SIRS‐negative patients. In all subpopulations, the algorithm demonstrates a strong ability to differentiate patients who satisfied the criteria for sepsis within 12 h of emergency department presentation from those who did not. AUROC, area under the receiver operating curve; ROC, receiver operating characteristic; SIRS, systemic inflammatory response syndrome; SOFA, Sequential Organ Failure Assessment

FIGURE 2

PR curves of the algorithm in the testing cohort for (a) all patients, (b) SOFA‐positive patients, and (c) SIRS‐negative patients. Recall (also known as sensitivity) is displayed on the x‐axis and precision (also known as positive predictive value) is displayed on the y‐axis. PR, precision recall; SIRS, systemic inflammatory response syndrome; SOFA, Sequential Organ Failure Assessment

FIGURE 3

Feature importance outputted by random forest. PCT and IL‐6 emerge as the most important features

FIGURE 4

Algorithm‐determined probability of sepsis for specific population subgroups in the testing cohort. (a) Subgroups of sepsis based on the Sepsis‐3 definition. (b) Subgroups of nonseptic patients based on the presence of infection and/or organ dysfunction (OR). In both subgroups, a trend of increasing probability with increasing disease severity emerges

FIGURE 5

Risk group analysis for three outcomes in the testing cohort: (a) length of hospital stay, (b) 30‐day mortality, (c) 30‐day inpatient readmission. For each risk group and outcome, survival estimates were generated based on the Kaplan‐Meier method, and comparisons of survival distributions were based on the log‐rank test. Statistically significant differences between each of the risk groups are observed for all three outcomes