Development and validation of a dynamic 48-hour in-hospital mortality risk stratification for COVID-19 in a UK teaching hospital: a retrospective cohort study

Abstract

Objectives: To develop a disease stratification model for COVID-19 that updates according to changes in a patient's condition while in hospital to facilitate patient management and resource allocation.

Design: In this retrospective cohort study, we adopted a landmarking approach to dynamic prediction of all-cause in-hospital mortality over the next 48 hours. We accounted for informative predictor missingness and selected predictors using penalised regression.

Setting: All data used in this study were obtained from a single UK teaching hospital.

Participants: We developed the model using 473 consecutive patients with COVID-19 presenting to a UK hospital between 1 March 2020 and 12 September 2020; and temporally validated using data on 1119 patients presenting between 13 September 2020 and 17 March 2021.

Primary and secondary outcome measures: The primary outcome is all-cause in-hospital mortality within 48 hours of the prediction time. We accounted for the competing risks of discharge from hospital alive and transfer to a tertiary intensive care unit for extracorporeal membrane oxygenation.

Results: Our final model includes age, Clinical Frailty Scale score, heart rate, respiratory rate, oxygen saturation/fractional inspired oxygen ratio, white cell count, presence of acidosis (pH <7.35) and interleukin-6. Internal validation achieved an area under the receiver operating characteristic (AUROC) of 0.90 (95% CI 0.87 to 0.93) and temporal validation gave an AUROC of 0.86 (95% CI 0.83 to 0.88).

Conclusions: Our model incorporates both static risk factors (eg, age) and evolving clinical and laboratory data, to provide a dynamic risk prediction model that adapts to both sudden and gradual changes in an individual patient's clinical condition. On successful external validation, the model has the potential to be a powerful clinical risk assessment tool.

Trial registration: The study is registered as 'researchregistry5464' on the Research Registry (www.researchregistry.com).

Keywords: COVID-19; epidemiology; risk management; statistics & research methods.

Figure 1

Performance metrics for in-hospital mortality in the training dataset. (A) Receiver operating characteristic plot, with labels indicating the corresponding threshold and the dashed line indicating the line of no discrimination. (B) Precision-recall plot, with the 2.8% observed incidence indicated by the dashed line. (C) NNE against sensitivity. (D) Calibration plot (with 95% CI), by tenths of predicted risk and a LOESS interpolation (grey), with the dashed line indicating perfect calibration. AUPRC, area under the precision-recall curve; AUROC, area under the receiver operating characteristic; FPR, false positive rate; LOESS, locally estimated scatterplot smoothing; NNE, number needed to evaluate; PPV, positive predictive value; TPR, true positive rate.

Figure 2

Performance metrics for in-hospital mortality in the validation dataset. (A) Receiver operating characteristic plot, with labels indicating the corresponding threshold and the dashed line indicating the line of no discrimination. (B) Precision-recall plot, with the 3.1% observed incidence indicated by the dashed line. (C) NNE against sensitivity. (D) Calibration plot (with 95% CI), by tenths of predicted risk and a LOESS interpolation (grey), with the dashed line indicating perfect calibration. AUPRC, area under the precision-recall curve; AUROC, area under the receiver operating characteristic; FPR, false positive rate; LOESS, locally estimated scatterplot smoothing; NNE, number needed to evaluate; PPV, positive predictive value; TPR, true positive rate.