Development and validation of early prediction models for new-onset functional impairment in patients after being transferred from the ICU

Abstract

A significant number of intensive care unit (ICU) survivors experience new-onset functional impairments that impede their activities of daily living (ADL). Currently, no effective assessment tools are available to identify these high-risk patients. This study aims to develop an interpretable machine learning (ML) model for predicting the onset of functional impairment in critically ill patients. Data for this study were sourced from a comprehensive hospital in China, focusing on adult patients admitted to the ICU from August 2022 to August 2023 without prior functional impairments. A least absolute shrinkage and selection operator (LASSO) model was utilized to select predictors for inclusion in the model. Four models, logistic regression, support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost), were constructed and validated. Model performance was assessed using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). Additionally, the DALEX package was employed to enhance the interpretability of the final models. The study ultimately included 1,380 patients, with 684 (49.6%) exhibiting new-onset functional impairment on the seventh day after leaving the ICU. Among the four models evaluated, the SVM model demonstrated the best performance, with an AUC of 0.909, accuracy of 0.838, sensitivity of 0.902, specificity of 0.772, PPV of 0.802, and NPV of 0.886. ML models are reliable tools for predicting new-onset functional impairments in critically ill patients. Notably, the SVM model emerged as the most effective, enabling early identification of patients at high risk and facilitating the implementation of timely interventions to improve ADL.

Keywords: Activities of daily living; Functional impairment; Intensive care unit; Machine learning; Prediction model.

Figure 1

Flowchart of patient selection. Abbreviations: ICU, intensive care unit.

Figure 2

Selection of predictors using LASSO regression.

Figure 3

Performance of the four models in the temporal validation dataset (A Receiver operating characteristic curves (ROC), B calibration plot).

Figure 4

Feature importance derived from support vector machine model. This figure is the result of the DALEX package. The X-axis represents the loss in the area under the curve (AUC) calculated after randomly permuting the feature compared to the original AUC. The greater this loss, the higher the model’s importance of this feature. CCI, Charlson Comorbidity Index; APACHE II score, Acute Physiology and Chronic Health Evaluation II score; CRP C-reactive protein; MV, mechanical ventilation.

Figure 5

This figure was made with the DALEX package to explain support vector machine model predictions. The numbers on the X-axis indicate the predicted probability of new-onset functional impairment obtained from the model's estimation based on a patient's features. CCI, Charlson Comorbidity Index; APACHE II score, Acute Physiology and Chronic Health Evaluation II score; MV, mechanical ventilation; CRP C-reactive protein.