Development and validation of machine learning-based risk prediction models for ICU-acquired weakness: a prospective cohort study

Abstract

Background: Intensive care unit (ICU)-acquired weakness (ICUAW) is a prevalent complication in critically ill patients, marked by symmetrical respiratory and limb muscle weakness, which adversely affects long-term outcomes. Early identification of high-risk patients and prevention are essential to mitigate its impact. Traditional risk prediction models, based on cohort data, have limitations in addressing the complex, non-linear relationships among diverse risk factors due to patient heterogeneity and the dynamic nature of critical illness. Machine learning offers a promising alternative by integrating heterogeneous data-clinical, laboratory, and physiological-to enhance predictive accuracy and individualization. Additionally, machine learning can identify novel risk factors and mechanisms overlooked by conventional methods, supporting early intervention and targeted prevention strategies to improve patient prognosis. Therefore, this study aims to develop and validate risk prediction models for ICUAW based on multiple machine learning algorithms.

Methods: Four machine learning algorithms were employed. Bedside ultrasound machines were used to assess ICUAW in patients admitted to the ICU twice, once within 24 hours of ICU admission and once on the 7th day of ICU admission. Eighteen features screened through a previous umbrella review informed the models. The performance of the models was evaluated based on multiple assessment metrics, such as the area under the receiver operating characteristic curve (AUC).

Results: A total of 749 patients were enrolled in the study, and 382 patients (51%) developed ICUAW. Specifically, 524 patients were assigned to the training set, and 225 patients were assigned to the internal validation set. Among the four machine-learning models, AUC ranged from 0.830 to 0.978. The eXtreme Gradient Boosting exhibited the best performance, achieving an AUC of 0.978 (95%CI 0.962-0.994), with 0.924 accuracy, 0.911 sensitivity, 0.941 specificity, 0.924 F1 score, and a Brier score of 0.084. The results of the Decision Curve Analysis also corroborate these results.

Conclusions: A machine learning prediction model can be developed, leveraging its robust learning capabilities to identify patients at high risk of developing ICUAW. This approach facilitates standardized management of ICUAW, thereby potentially reducing its incidence.

Keywords: Intensive care unit; Machine learning; Muscle weakness; Risk factors; Risk prediction.

Fig.1

A: Receiver operating characteristic curves in the training set; B: Receiver operating characteristic curves in the validation set; AUC, areas under the receiver operating characteristic curves; XGBoost: eXtreme Gradient Boosting; LR: Logistic Regression; GNB: Gaussian Naive Bayes; SVM: Support Vector Machine

Fig. 2

A: Calibration curves in the training set; B: Calibration curves in the validation set; GNB: Gaussian Naive Bayes; LR: Logistic Regression; SVM: Support Vector Machine; XGBoost: eXtreme Gradient Boosting

Fig. 3

A: Clinical decision curves in the training set; B: Clinical decision curves in the validation set; XGBoost: eXtreme Gradient Boosting; LR: Logistic regression; GNB: Gaussian Naive Bayes; SVM: Support Vector Machine

Fig. 4

SHAP variable importance chart. APACHE II: acute physiology and chronic health evaluation II; MODS: multiple organ dysfunction syndrome; SIRS: systemic inflammatory response syndrome