Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul:159:274-288.
doi: 10.1016/j.jclinepi.2023.04.011. Epub 2023 May 2.

Validity of prognostic models of critical COVID-19 is variable. A systematic review with external validation

Gabriela Cárdenas-Fuentes  1 , Magda Bosch de Basea  2 , Inés Cobo  2 , Isaac Subirana  3 , Mario Ceresa  4 , Ernest Famada  5 , Elena Gimeno-Santos  6 , Laura Delgado-Ortiz  2 , Rosa Faner  7 , María Molina-Molina  8 , Àlvar Agustí  9 , Xavier Muñoz  10 , Oriol Sibila  11 , Joaquim Gea  12 , Judith Garcia-Aymerich  2
Affiliations

Validity of prognostic models of critical COVID-19 is variable. A systematic review with external validation

Gabriela Cárdenas-Fuentes et al. J Clin Epidemiol. 2023 Jul.

Abstract

Objectives: To identify prognostic models which estimate the risk of critical COVID-19 in hospitalized patients and to assess their validation properties.

Study design and setting: We conducted a systematic review in Medline (up to January 2021) of studies developing or updating a model that estimated the risk of critical COVID-19, defined as death, admission to intensive care unit, and/or use of mechanical ventilation during admission. Models were validated in two datasets with different backgrounds (HM [private Spanish hospital network], n = 1,753, and ICS [public Catalan health system], n = 1,104), by assessing discrimination (area under the curve [AUC]) and calibration (plots).

Results: We validated 18 prognostic models. Discrimination was good in nine of them (AUCs ≥ 80%) and higher in those predicting mortality (AUCs 65%-87%) than those predicting intensive care unit admission or a composite outcome (AUCs 53%-78%). Calibration was poor in all models providing outcome's probabilities and good in four models providing a point-based score. These four models used mortality as outcome and included age, oxygen saturation, and C-reactive protein among their predictors.

Conclusion: The validity of models predicting critical COVID-19 by using only routinely collected predictors is variable. Four models showed good discrimination and calibration when externally validated and are recommended for their use.

Keywords: COVID-19; Critical disease; Epidemiology; External validation; Intensive care unit; Prognostic models.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
PRISMA flow diagram.
Fig. 2
Fig. 2
Calibration of six critical COVID-19 prognostic models providing a point-based scoring system in the HM dataset. The dashed black line indicates the LOESS fit. The five imputed datasets are differentiated in colors.
Fig. 3
Fig. 3
Calibration of four critical COVID-19 prognostic models providing a point-based scoring system in the ICS dataset. The dashed black line indicates the LOESS fit. The five imputed datasets are differentiated in colors.
See this image and copyright information in PMC

References

    1. Islam N., Shkolnikov V.M., Acosta R.J., Klimkin I., Kawachi I., Irizarry R.A., et al. Excess deaths associated with covid-19 pandemic in 2020: age and sex disaggregated time series analysis in 29 high income countries. BMJ. 2021;373:n1137. - PMC - PubMed
    1. Wu Z., McGoogan J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72314 cases from the Chinese center for disease control and prevention. JAMA. 2020;323:1239–1242. - PubMed
    1. European centre for disease prevention and control. COVID-19 situation update worldwide, as of week 14, updated 14 April 2022. 2020. https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases Available at:
    1. Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369:m1328. - PMC - PubMed
    1. Moons K.G.M., Wolff R.F., Riley R.D., Whiting P.F., Westwood M., Collins G.S., et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med. 2019;170:W1–W33. - PubMed

Publication types