The influence of the dynamic context of the pandemic on the predictive performance of mortality predictions over time in older patients hospitalized for COVID-19

Abstract

Objectives: During the COVID-19 pandemic, dynamic factors, such as governmental policies, improved treatment, prevention options, and viral mutations changed the incidence of outcomes and possibly changed the relation between predictors and outcomes. The aim of the present study was to assess whether the dynamic context of the pandemic influenced the predictive performance of mortality predictions over time in older patients hospitalized for COVID-19.

Study design and setting: The COVID-19 Ouderen Landelijke Database study, a multicentre cohort study in the Netherlands, included COVID-19 patients aged 70 years and older hospitalized during the first (early 2020), second (late 2020), third (late 2021), or fourth wave (early 2022). We developed a prediction model for in-hospital mortality that included variables commonly collected at the emergency department with least absolute shrinkage and selection operator (LASSO) regression on patients admitted in the first pandemic wave and temporally validated this model in patients admitted in the second, third, or fourth wave.

Results: In total, 3067 patients (median age 79 years, 60% men) were included. The final model included demographics, frailty, and indicators of disease severity that were generally available within 3 hours after admission. The model differentiated between death and alive after hospitalization for COVID-19 with an area under the curve (AUC) of 0.80 (95% CI: 0.76-0.84) in the internal validation cohort. In terms of discrimination and calibration, predictive performance of the model decreased over time with an AUC of 0.76 (0.73-0.79) and calibration slope of 0.81 (0.68-0.96) in the second wave, an AUC of 0.77 (0.72-0.82) and calibration slope of 0.85 (0.65-1.10) in the third wave, and an AUC of 0.59 (0.48-0.70) and calibration slope of 0.35 (-0.05, 0.72) in the fourth wave.

Conclusion: Compared to the moderate model performance in the first wave, we observed a slight decrease in terms of discrimination and calibration in the second and third wave with a much larger decrease in the fourth wave. This highlights the importance of ongoing data collection, monitoring of model performance, and model updates during a pandemic.

Keywords: COVID-19; In-hospital mortality; Older people; Pandemic; Prediction model; Temporal validation.