External Validation of Mortality Scores among High-Risk COVID-19 Patients: A Romanian Retrospective Study in the First Pandemic Year

Abstract

Background: We aimed to externally validate three prognostic scores for COVID-19: the 4C Mortality Score (4CM Score), the COVID-GRAM Critical Illness Risk Score (COVID-GRAM), and COVIDAnalytics.

Methods: We evaluated the scores in a retrospective study on adult patients hospitalized with severe/critical COVID-19 (1 March 2020-1 March 2021), in the Teaching Hospital of Infectious Diseases, Cluj-Napoca, Romania. We assessed all the deceased patients matched with two survivors by age, gender, and at least two comorbidities. The areas under the receiver-operating characteristic curves (AUROCs) were computed for in-hospital mortality.

Results: Among 780 severe/critical COVID-19 patients, 178 (22.8%) died. We included 474 patients according to the case definition (158 deceased/316 survivors). The median age was 75 years; diabetes mellitus, malignancies, chronic pulmonary diseases, and chronic kidney and moderate/severe liver diseases were associated with higher risks of death. According to the predefined 4CM Score, the mortality rates were 0% (low), 13% (intermediate), 27% (high), and 61% (very high). The AUROC for the 4CM Score was 0.72 (95% CI: 0.67-0.77) for in-hospital mortality, close to COVID-GRAM, with slightly greater discriminatory ability for COVIDAnalytics: 0.76 (95% CI: 0.71-0.80).

Conclusion: All the prognostic scores showed close values compared to their validation cohorts, were fairly accurate in predicting mortality, and can be used to prioritize care and resources.

Keywords: COVID-19; Romania; comorbidities; mortality scores; validation.

Figure 1

Receiver-operating curves (ROCs) comparing the original 4CM Score (dotted green line), COVID-GRAM (dashed blue), and COVIDAnalytics scores (dashed purple).

Figure 2

The parameters of the three mortality prediction scores for all the cut-offs. (A)—4CM Score, (B)—COVID-GRAM, and (C)—COVIDAnalytics. Black line: diagnostic accuracy; solid blue line: sensitivity; dashed blue line: negative predictive value; solid red line: specificity; dashed red line: positive predictive value.

Figure 3

Death rate (left y axis) and distribution (right y axis) of the original 4CM Scores. Dot sizes are proportional to the number of patients with each score value, with the inner red dots showing the fatal cases. The smooth blue line shows the predicted death rate according to the logistic model (GLM).