Mortality prediction in intensive care units including premorbid functional status improved performance and internal validity

Abstract

Objective: Prognostic models are key for benchmarking intensive care units (ICUs). They require up-to-date predictors and should report transportability properties for reliable predictions. We developed and validated an in-hospital mortality risk prediction model to facilitate benchmarking, quality assurance, and health economics evaluation.

Study design and setting: We retrieved data from the database of an international (Finland, Estonia, Switzerland) multicenter ICU cohort study from 2015 to 2017. We used a hierarchical logistic regression model that included age, a modified Simplified Acute Physiology Score-II, admission type, premorbid functional status, and diagnosis as grouping variable. We used pooled and meta-analytic cross-validation approaches to assess temporal and geographical transportability.

Results: We included 61,224 patients treated in the ICU (hospital mortality 10.6%). The developed prediction model had an area under the receiver operating characteristic curve 0.886, 95% confidence interval (CI) 0.882-0.890; a calibration slope 1.01, 95% CI (0.99-1.03); a mean calibration -0.004, 95% CI (-0.035 to 0.027). Although the model showed very good internal validity and geographic discrimination transportability, we found substantial heterogeneity of performance measures between ICUs (I-squared: 53.4-84.7%).

Conclusion: A novel framework evaluating the performance of our prediction model provided key information to judge the validity of our model and its adaptation for future use.

Keywords: Case mix; Intensive care; In–hospital mortality; Prediction model; Transportability; Validation.