Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2024 Nov 1;14(1):26344.
doi: 10.1038/s41598-024-70333-6.

Boosting the accuracy of existing models by updating and extending: using a multicenter COVID-19 ICU cohort as a proxy

Daniek A M Meijs  1   2   3 , Laure Wynants  4   5   6 , Sander M J van Kuijk  7 , Clarissa I E Scheeren  8 , Anisa Hana  9   10 , Jannet Mehagnoul-Schipper  11 , Björn Stessel  12   13 , Margot Vander Laenen  14 , Eline G M Cox  15   16 , Jan-Willem E M Sels  15   17 , Luc J M Smits  4 , Johannes Bickenbach  18 , Dieter Mesotten  13   14 , Iwan C C van der Horst  15   19 , Gernot Marx  18 , Bas C T van Bussel  15   19   4 ; CoDaP Investigators
Collaborators, Affiliations
Multicenter Study

Boosting the accuracy of existing models by updating and extending: using a multicenter COVID-19 ICU cohort as a proxy

Daniek A M Meijs et al. Sci Rep. .

Abstract

Most published prediction models for Coronavirus Disease 2019 (COVID-19) were poorly reported, at high risk of bias, and heterogeneous in model performance. To tackle methodological challenges faced in previous prediction studies, we investigated whether model updating and extending improves mortality prediction, using the Intensive Care Unit (ICU) as a proxy. All COVID-19 patients admitted to seven ICUs in the Euregio-Meuse Rhine during the first pandemic wave were included. The 4C Mortality and SEIMC scores were selected as promising prognostic models from an external validation study. Five predictors could be estimated based on cohort size. TRIPOD guidelines were followed and logistic regression analyses with the linear predictor, APACHE II score, and country were performed. Bootstrapping with backward selection was applied to select variables for the final model. Additionally, shrinkage was performed. Model discrimination was displayed as optimism-corrected areas under the ROC curve and calibration by calibration slopes and plots. The mortality rate of the 551 included patients was 36%. Discrimination of the 4C Mortality and SEIMC scores increased from 0.70 to 0.74 and 0.70 to 0.73 and calibration plots improved compared to the original models after updating and extending. Mortality prediction can be improved after updating and extending of promising models.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Figure 1
Figure 1
Flowchart EICC cohort,,.
Figure 2
Figure 2
Calibration plots of the original compared to the updated and extended scores. The predicted probability of mortality is reported on the x-axis and the actual (observed) probability on the y-axis. The diagonal reflects optimal calibration and above the x-axis, histograms of predicted risk are illustrated for patients who died and survived in the ICU. In addition, 95% confidence boundaries are shown by the grey area.
Figure 3
Figure 3
The prediction process. This visual representation conveys the sequential steps involved in the prediction process.
See this image and copyright information in PMC

References

    1. Ma, X. & Vervoort, D. Critical care capacity during the COVID-19 pandemic: Global availability of intensive care beds. J. Crit. Care58, 96–97. 10.1016/j.jcrc.2020.04.012 (2020). - PMC - PubMed
    1. Douin, D. J. et al. ICU bed utilization during the Coronavirus Disease 2019 pandemic in a multistate analysis-March to June 2020. Crit. Care Explor.3, e0361. 10.1097/CCE.0000000000000361 (2021). - PMC - PubMed
    1. Wynants, L. et al. Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal. BMJ369, m1328. 10.1136/bmj.m1328 (2020). - PMC - PubMed
    1. de Jong, V. M. T. et al. Clinical prediction models for mortality in patients with covid-19: External validation and individual participant data meta-analysis. BMJ378, e069881. 10.1136/bmj-2021-069881 (2022). - PMC - PubMed
    1. Moons, K. G. et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart98, 691–698. 10.1136/heartjnl-2011-301247 (2012). - PubMed

Publication types

LinkOut - more resources