Prediction of posttraumatic functional recovery in middle-aged and older patients through dynamic ensemble selection modeling

Abstract

Introduction: Age-specific risk factors may delay posttraumatic functional recovery; complex interactions exist between these factors. In this study, we investigated the prediction ability of machine learning models for posttraumatic (6 months) functional recovery in middle-aged and older patients on the basis of their preexisting health conditions.

Methods: Data obtained from injured patients aged ≥45 years were divided into training-validation (n = 368) and test (n = 159) data sets. The input features were the sociodemographic characteristics and baseline health conditions of the patients. The output feature was functional status 6 months after injury; this was assessed using the Barthel Index (BI). On the basis of their BI scores, the patients were categorized into functionally independent (BI >60) and functionally dependent (BI ≤60) groups. The permutation feature importance method was used for feature selection. Six algorithms were validated through cross-validation with hyperparameter optimization. The algorithms exhibiting satisfactory performance were subjected to bagging to construct stacking, voting, and dynamic ensemble selection models. The best model was evaluated on the test data set. Partial dependence (PD) and individual conditional expectation (ICE) plots were created.

Results: In total, nineteen of twenty-seven features were selected. Logistic regression, linear discrimination analysis, and Gaussian Naive Bayes algorithms exhibited satisfactory performances and were, therefore, used to construct ensemble models. The k-Nearest Oracle Elimination model outperformed the other models when evaluated on the training-validation data set (sensitivity: 0.732, 95% CI: 0.702-0.761; specificity: 0.813, 95% CI: 0.805-0.822); it exhibited compatible performance on the test data set (sensitivity: 0.779, 95% CI: 0.559-0.950; specificity: 0.859, 95% CI: 0.799-0.912). The PD and ICE plots showed consistent patterns with practical tendencies.

Conclusion: Preexisting health conditions can predict long-term functional outcomes in injured middle-aged and older patients, thus predicting prognosis and facilitating clinical decision-making.

Keywords: dynamic ensemble selection; machine learning; middle-aged patient; older patient; traumatic injury.

Figure 1

Machine learning process. Data obtained from the patients were first checked for missing values and outliners and then divided into training–validation data set (for model construction) and independent test (for model evaluation) data sets. The permutation feature importance method was used for feature selection. Because our data were imbalanced, random oversampling was applied on each training fold during cross-validation. Six algorithms, including support vector machine (SVM), logistic regression (LR), k-nearest neighbor (KNN), Gaussian Naive Bayes (GNB), linear discrimination analysis (LDA), and decision tree (DT), were validated through five-fold cross-validation. The algorithms exhibiting satisfactory performance (LR, GNB, and LDA) were subjected to bagging; afterward, they were used as the pool of classifiers for ensemble models (i.e., stacking, voting, and dynamic ensemble selection [DES] models). The ensemble models were compared in terms of performance, and the best model (k-nearest oracle elimination [KNORA-E]) was evaluated on the independent test data set. Partial dependence (PD) and individual conditional expectation (ICE) plots were constructed to investigate the interpretability of the model.

Figure 2

Permutation feature importance selection. The figure presents important scores of the features assessed using the permutation importance ranking method (1,000 repeats). CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disorder; HOSP_ISS, injury severity score during hospitalization, ED_ISS, injury severity score upon admission to the emergency department; CCI score, Charlson comorbidity index score; RTS, revised traumatic score; EDU, education level; ISS, injury severity score before discharge; DM, diabetes mellitus; LIVER_DIS, liver diseases; HF, heart failure; HT, hypertension; HIP_FX, hip bone fracture; CFS_BASE, baseline score on the Clinical Frailty Scale; and BI_BASE, baseline score on the Barthel Index.

Figure 3

Area under the receiver operating characteristic curve (ROC-AUC) of the k-nearest oracle elimination (KNORA-E) model. (A) ROC-AUC of the KNORA-E model evaluated using the cross-validation data set. (B) ROC-AUC of the KNORA-E model evaluated using the test data set.

Figure 4

Partial dependence (PD) plots (blue lines) with individual conditional expectation (ICE) plots (grey lines). The plots indicate the overall (for PD) and individual (for ICE) effects of the features, including the baseline Barthel Index (BI_BASE) score, baseline Clinical Frailty Scale (CFS_BASE) score, revised traumatic score (RTS), and injury severity score (ISS).