Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov 23:10:1284081.
doi: 10.3389/fmed.2023.1284081. eCollection 2023.

Development and validation of a machine learning model integrated with the clinical workflow for early detection of sepsis

Mohammed A Mahyoub  1   2 , Ravi R Yadav  1 , Kacie Dougherty  1 , Ajit Shukla  1
Affiliations

Development and validation of a machine learning model integrated with the clinical workflow for early detection of sepsis

Mohammed A Mahyoub et al. Front Med (Lausanne). .

Abstract

Background: Sepsis is a life-threatening condition caused by a dysregulated response to infection, affecting millions of people worldwide. Early diagnosis and treatment are critical for managing sepsis and reducing morbidity and mortality rates.

Materials and methods: A systematic design approach was employed to build a model that predicts sepsis, incorporating clinical feedback to identify relevant data elements. XGBoost was utilized for prediction, and interpretability was achieved through the application of Shapley values. The model was successfully deployed within a widely used Electronic Medical Record (EMR) system.

Results: The developed model demonstrated robust performance pre-operations, with a sensitivity of 92%, specificity of 93%, and a false positive rate of 7%. Following deployment, the model maintained comparable performance, with a sensitivity of 91% and specificity of 94%. Notably, the post-deployment false positive rate of 6% represents a substantial reduction compared to the currently deployed commercial model in the same health system, which exhibits a false positive rate of 30%.

Discussion: These findings underscore the effectiveness and potential value of the developed model in improving timely sepsis detection and reducing unnecessary alerts in clinical practice. Further investigations should focus on its long-term generalizability and impact on patient outcomes.

Keywords: XGBoost; early detection; machine learning; machine learning deployment; model interpretability; sepsis.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Research methodology workflow.
Figure 2
Figure 2
Model deployment and operationalization.
Figure 3
Figure 3
Confusion matrix of the sepsis prediction model. Sensitivity was 92% and specificity was 93% with a 7% false positive rate. Colorbar represents the count of cases.
Figure 4
Figure 4
Receiver operating curve of the sepsis prediction model: pre-deployment performance.
Figure 5
Figure 5
Precision-recall curve of the sepsis prediction model: pre-deployment performance.
Figure 6
Figure 6
Highly important features and their corresponding Shapley values. The grey color depicts the missing values (NaN) in a certain feature.
Figure 7
Figure 7
Operations performance of the sepsis prediction model. Sensitivity was 91% and specificity was 94% at a 0.05 risk threshold.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. . The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. (2016) 315:801–10. doi: 10.1001/jama.2016.0287, PMID: - DOI - PMC - PubMed
    1. Fleischmann C, Scherag A, Adhikari NKJ, Hartog CS, Tsaganos T, Schlattmann P, et al. . Assessment of global incidence and mortality of hospital-treated Sepsis. Current estimates and limitations. Am J Respir Crit Care Med. (2016) 193:259–72. doi: 10.1164/rccm.201504-0781OC, PMID: - DOI - PubMed
    1. Fleischmann-Struzek C, Mellhammar L, Rose N, Cassini A, Rudd KE, Schlattmann P, et al. . Incidence and mortality of hospital-and ICU-treated sepsis: results from an updated and expanded systematic review and meta-analysis. Intensive Care Med. (2020) 46:1552–62. doi: 10.1007/s00134-020-06151-x, PMID: - DOI - PMC - PubMed
    1. Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. . Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the global burden of disease study. Lancet. (2020) 395:200–11. doi: 10.1016/S0140-6736(19)32989-7, PMID: - DOI - PMC - PubMed
    1. Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the emergency department. Am J Emerg Med. (2019) 37:1490–7. doi: 10.1016/j.ajem.2018.10.058, PMID: - DOI - PubMed

LinkOut - more resources