External validation of the 4C Mortality Score for hospitalised patients with COVID-19 in the RECOVER network

Abstract

Objectives: Estimating mortality risk in hospitalised SARS-CoV-2+ patients may help with choosing level of care and discussions with patients. The Coronavirus Clinical Characterisation Consortium Mortality Score (4C Score) is a promising COVID-19 mortality risk model. We examined the association of risk factors with 30-day mortality in hospitalised, full-code SARS-CoV-2+ patients and investigated the discrimination and calibration of the 4C Score. This was a retrospective cohort study of SARS-CoV-2+ hospitalised patients within the RECOVER (REgistry of suspected COVID-19 in EmeRgency care) network.

Setting: 99 emergency departments (EDs) across the USA.

Participants: Patients ≥18 years old, positive for SARS-CoV-2 in the ED, and hospitalised.

Primary outcome: Death within 30 days of the index visit. We performed logistic regression analysis, reporting multivariable risk ratios (MVRRs) and calculated the area under the ROC curve (AUROC) and mean prediction error for the original 4C Score and after dropping the C reactive protein (CRP) component.

Results: Of 6802 hospitalised patients with COVID-19, 1149 (16.9%) died within 30 days. The 30-day mortality was increased with age 80+ years (MVRR=5.79, 95% CI 4.23 to 7.34); male sex (MVRR=1.17, 1.05 to 1.28); and nursing home/assisted living facility residence (MVRR=1.29, 1.1 to 1.48). The 4C Score had comparable discrimination in the RECOVER dataset compared with the original 4C validation dataset (AUROC: RECOVER 0.786 (95% CI 0.773 to 0.799), 4C validation 0.763 (95% CI 0.757 to 0.769). Score-specific mortalities in our sample were lower than in the 4C validation sample (mean prediction error 6.0%). Dropping the CRP component from the 4C Score did not substantially affect discrimination and 4C risk estimates were now close (mean prediction error 0.7%).

Conclusions: We independently validated 4C Score as predicting risk of 30-day mortality in hospitalised SARS-CoV-2+ patients. We recommend dropping the CRP component of the score and using our recalibrated mortality risk estimates.

Keywords: ACCIDENT & EMERGENCY MEDICINE; Adult intensive & critical care; COVID-19; EPIDEMIOLOGY; GENERAL MEDICINE (see Internal Medicine).

Figure 1

Comparison of 4C (Coronavirus Clinical Characterisation Consortium) validation and RECOVER datasets. (A) 4C Mortality Scores were lower in the RECOVER dataset than in the original 4C validation dataset. (B) ROC curves for the 4C Mortality Score (categorised into the nine ranges from (A)) in the 4C validation dataset and the RECOVER dataset. (C) Calibration plot (modified Bland-Altman) showing prediction error versus observed mortality for the 4C Mortality Score with and without the C reactive protein (CRP) component. Points from left to right are in the 4C Mortality Score ranges shown in figure (A) from left to right. AUROC, area under the ROC curve; RECOVER, REgistry of suspected COVID-19 in EmeRgency care.