Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Clinical Trial
. 2020 Dec 9;15(12):e0243262.
doi: 10.1371/journal.pone.0243262. eCollection 2020.

COVID-19 mortality risk assessment: An international multi-center study

Dimitris Bertsimas  1   2 , Galit Lukin  2 , Luca Mingardi  1   2 , Omid Nohadani  3 , Agni Orfanoudaki  2 , Bartolomeo Stellato  1   2 , Holly Wiberg  2 , Sara Gonzalez-Garcia  4 , Carlos Luis Parra-Calderón  4 , Kenneth Robinson  5 , Michelle Schneider  5 , Barry Stein  5 , Alberto Estirado  6 , Lia A Beccara  7 , Rosario Canino  7 , Martina Dal Bello  8 , Federica Pezzetti  7 , Angelo Pan  7 ; Hellenic COVID-19 Study Group
Affiliations
Clinical Trial

COVID-19 mortality risk assessment: An international multi-center study

Dimitris Bertsimas et al. PLoS One. .

Abstract

Timely identification of COVID-19 patients at high risk of mortality can significantly improve patient management and resource allocation within hospitals. This study seeks to develop and validate a data-driven personalized mortality risk calculator for hospitalized COVID-19 patients. De-identified data was obtained for 3,927 COVID-19 positive patients from six independent centers, comprising 33 different hospitals. Demographic, clinical, and laboratory variables were collected at hospital admission. The COVID-19 Mortality Risk (CMR) tool was developed using the XGBoost algorithm to predict mortality. Its discrimination performance was subsequently evaluated on three validation cohorts. The derivation cohort of 3,062 patients has an observed mortality rate of 26.84%. Increased age, decreased oxygen saturation (≤ 93%), elevated levels of C-reactive protein (≥ 130 mg/L), blood urea nitrogen (≥ 18 mg/dL), and blood creatinine (≥ 1.2 mg/dL) were identified as primary risk factors, validating clinical findings. The model obtains out-of-sample AUCs of 0.90 (95% CI, 0.87-0.94) on the derivation cohort. In the validation cohorts, the model obtains AUCs of 0.92 (95% CI, 0.88-0.95) on Seville patients, 0.87 (95% CI, 0.84-0.91) on Hellenic COVID-19 Study Group patients, and 0.81 (95% CI, 0.76-0.85) on Hartford Hospital patients. The CMR tool is available as an online application at covidanalytics.io/mortality_calculator and is currently in clinical use. The CMR model leverages machine learning to generate accurate mortality predictions using commonly available clinical features. This is the first risk score trained and validated on a cohort of COVID-19 patients from Europe and the United States.

PubMed Disclaimer

Conflict of interest statement

KN, MS, BS are employed by Hartford HealthCare. ON is employed by Benefits Science Technologies. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1
Fig 1. SHAP importance plots for final model.
The top 10 features are displayed in panel (a), ordered by decreasing significance. For a given feature, the corresponding row indicates the SHAP values as the feature ranges from its lowest (blue) to highest (red) value. Panel (b)-(j) display the individual feature plots and the impact of each feature on the mortality risk (colors indicate the age here) with gray areas indicating reference ranges.
Fig 2
Fig 2. Visualization of the calculator interface.
Using the SHAP package, personalized interpretations of the predicted score are provided to the user.
See this image and copyright information in PMC

References

    1. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020;20: 533–534. 10.1016/S1473-3099(20)30120-1 - DOI - PMC - PubMed
    1. Alimadadi A, Aryal S, Manandhar I, Munroe PB, Joe B, Cheng X. Artificial intelligence and machine learning to fight COVID-19. Physiol Genomics. 2020;52: 200–202. 10.1152/physiolgenomics.00029.2020 - DOI - PMC - PubMed
    1. Chen T, Guestrin C. XGBoost: A scalable tree boosting system. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016. pp. 785–794.
    1. Pourhomayoun M, Shakibi M. Predicting Mortality Risk in Patients with COVID-19 Using Artificial Intelligence to Help Medical Decision-Making. medRxiv. 2020; 2020.03.30.20047308. 10.1101/2020.03.30.20047308 - DOI - PMC - PubMed
    1. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and Validation of a Clinical Risk Score to Predict the Occurrence of Critical Illness in Hospitalized Patients With COVID-19. JAMA Intern Med. 2020. 10.1001/jamainternmed.2020.2033 - DOI - PMC - PubMed

Publication types