A machine learning-based prediction model for in-hospital mortality among critically ill patients with hip fracture: An internal and external validated study

Abstract

Introduction: Few studies have investigated the in-hospital mortality among critically ill patients with hip fracture. This study aimed to develop and validate a model to estimate the risk of in-hospital mortality among critically ill patients with hip fracture.

Methods: For this study, data from the Medical Information Mart for Intensive Care III (MIMIC-III) Database and electronic Intensive Care Unit (eICU) Collaborative Research Database were evaluated. Enrolled patients (n=391) in the MIMIC-III database were divided into a training (2/3, n=260) and a validation (1/3, n=131) group at random. Using machine learning algorithms such as random forest, gradient boosting machine, decision tree, and eXGBoosting machine approach, the training group was utilized to train and optimize models. The validation group was used to internally validate models and the optimal model could be obtained in terms of discrimination (area under the receiver operating characteristic curve, AUROC) and calibration (calibration curve). External validation was done in the eICU Collaborative Research Database (n=165). To encourage practical use of the model, a web-based calculator was developed according to the eXGBoosting machine approach.

Results: The in-hospital death rate was 13.81% (54/391) in the MIMIC-III database and 10.91% (18/165) in the eICU Collaborative Research Database. Age, gender, anemia, mechanical ventilation, cardiac arrest, and chronic airway obstruction were the six model parameters which were identified using the Least Absolute Shrinkage and Selection Operator (LASSO) method combined with 10-fold cross-validation. The model established using the eXGBoosting machine approach showed the highest area under curve (AUC) value (0.797, 95% CI: 0.696-0.898) and the best calibrating ability, with a calibration slope of 0.999 and intercept of -0.019. External validation also revealed favorable discrimination (AUC: 0.715, 95% CI: 0.566-0.864; accuracy: 0.788) and calibration (calibration slope: 0.805) in the eICU Collaborative Research Database. The web-based calculator could be available at https://doctorwangsj-webcalculator-main-yw69yd.streamlitapp.com/.

Conclusion: The model has the potential to be a pragmatic risk prediction tool that is able to identify hip fracture patients who are at a high risk of in-hospital mortality in ICU settings, guide patient risk counseling, and simplify prognosis bench-marking by controlling for baseline risk.

Keywords: Critically ill patients; Hip fracture; In-hospital mortality; Machine learning; Prediction model.