Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Apr 10;78(4):889-899.
doi: 10.1093/cid/ciad618.

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

Katharina S Appel  1 , Ramsia Geisler  1 , Daniel Maier  1   2 , Olga Miljukov  3 , Sina M Hopff  4 , J Janne Vehreschild  1   5   6
Affiliations

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

Katharina S Appel et al. Clin Infect Dis. .

Abstract

Background: Numerous prognostic scores have been published to support risk stratification for patients with coronavirus disease 2019 (COVID-19).

Methods: We performed a systematic review to identify the scores for confirmed or clinically assumed COVID-19 cases. An in-depth assessment and risk of bias (ROB) analysis (Prediction model Risk Of Bias ASsessment Tool [PROBAST]) was conducted for scores fulfilling predefined criteria ([I] area under the curve [AUC)] ≥ 0.75; [II] a separate validation cohort present; [III] training data from a multicenter setting [≥2 centers]; [IV] point-scale scoring system).

Results: Out of 1522 studies extracted from MEDLINE/Web of Science (20/02/2023), we identified 242 scores for COVID-19 outcome prognosis (mortality 109, severity 116, hospitalization 14, long-term sequelae 3). Most scores were developed using retrospective (75.2%) or single-center (57.1%) cohorts. Predictor analysis revealed the primary use of laboratory data and sociodemographic information in mortality and severity scores. Forty-nine scores were included in the in-depth analysis. The results indicated heterogeneous quality and predictor selection, with only five scores featuring low ROB. Among those, based on the number and heterogeneity of validation studies, only the 4C Mortality Score can be recommended for clinical application so far.

Conclusions: The application and translation of most existing COVID scores appear unreliable. Guided development and predictor selection would have improved the generalizability of the scores and may enhance pandemic preparedness in the future.

Keywords: COVID-19; pandemic preparedness; prediction models; predictors; scores.

PubMed Disclaimer

Conflict of interest statement

Potential conflicts of interest. J. J. V. received research funding and consulting fees from Merck/MSD, Gilead, Pfizer, Astellas Pharma, Basilea, German Centre for Infection Research (DZIF), University Hospital Freiburg/Congress and Communication, Academy for Infectious Medicine, University Manchester, German Society for Infectious Diseases (DGI), German Federal Ministry of Education and Research (BMBF), Deutsches Zentrum für Luft- und Raumfahrt (DLR), University of Bristol, Rigshospitalet Copenhagen; consulting fees from Ärztekammer Nordrhein, University Hospital Aachen, Back Bay Strategies, German Society for Internal Medicine (DGIM), Shionogi, Molecular Health, Netzwerk Universitätsmedizin, Janssen, NordForsk, Biontech, APOGEPHA; and reports participation on Data Safety Monitoring or Advisory Boards for APOGEPHA Arzneimittel, Biontech, Merck/MSD, Gilead, and Pfizer. D. M. received speaker honoraria from Free University of Berlin and travel compensation from IQVIA. All other authors report no potential conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Figures

Figure 1.
Figure 1.
PRISMA flow chart. *Not every study had a DOI or had multiple DOIs. Abbreviations: COVID-19, coronavirus disease 2019; DOI, digital object identifier; PRISMA, Preferred Reporting Items for Systematic reviews and Meta-Analyses.
Figure 2.
Figure 2.
Relationship between predictor frequency within all scores (irrespective of category) and AUC of the overall score. The predictors were grouped by predictor type. Only predictors that were integrated at least twice in a score are presented. Abbreviations: AUC, area under the curve; COVID-19, coronavirus disease 2019.
Figure 3.
Figure 3.
Predictor composition aggregated by predictor type for all scores assigned to category 1 stratified by the level of selection. A, Level 1, B, Level 2. The AUC is displayed on the right y-axis. The sorting of the scores is determined by (I) the absolute number of categories and (II) the relative proportion across all scores. The color gradient from green to blue indicates the availability of the category, although in case of doubt, this also depends on the level of care. Similar presentations of scores assigned to categories 2–4 are displayed in the Supplementary Material. Abbreviations: AUC, area under the curve; COVID-19, coronavirus disease 2019.
Figure 4.
Figure 4.
Results of the ROB assessment using PROBAST. Abbreviations: ED, emergency department; ICU, intensive care unit; ROB, risk of bias.
See this image and copyright information in PMC

References

    1. Filip R, Puscaselu RG, Anchidin-Norocel L, Dimian M, Savage WK. Global challenges to public health care systems during the COVID-19 pandemic: a review of pandemic measures and problems. J Pers Med 2022; 12:1295. - PMC - PubMed
    1. Biancolella M, Colona VL, Mehrian-Shai R, et al. COVID-19 2022 update: transition of the pandemic to the endemic phase. Hum Genomics 2022; 16:19. - PMC - PubMed
    1. Shrestha LB, Foster C, Rawlinson W, Tedla N, Bull RA. Evolution of the SARS-CoV-2 Omicron variants BA.1 to BA.5: implications for immune escape and transmission. Rev Med Virol 2022; 32:e2381. - PMC - PubMed
    1. Therapeutics and COVID-19: living guideline, 13 January 2023. Geneva: World Health Organization, 2023. (WHO/2019-nCoV/therapeutics/2023.1). Licence: CC BY-NC-SA 3.0 IGO. - PubMed
    1. Long B, Carius BM, Chavez S, et al. Clinical update on COVID-19 for the emergency clinician: presentation and evaluation. Am J Emerg Med 2022; 54:46–57. - PMC - PubMed

Publication types