Mortality prediction in ICU Using a Stacked Ensemble Model

Abstract

Artificial intelligence (AI) technology has huge scope in developing models to predict the survival rate of critically ill patients in the intensive care unit (ICU). The availability of electronic clinical data has led to the widespread use of various machine learning approaches in this field. Innovative algorithms play a crucial role in boosting the performance of models. This study uses a stacked ensemble model to predict mortality in ICU by incorporating the clinical severity scoring results, in which several machine learning algorithms are employed to compare the performance. The experimental results show that the stacked ensemble model achieves good performance compared with the model without integrating the severity scoring results, which has the area under curve (AUC) of 0.879 and 0.862, respectively. To improve the performance of prediction, two feature subsets are obtained based on different feature selection techniques, labeled as SetS and SetT. Evaluation performances show that the SEM based on the SetS achieves a higher AUC value (0.879 and 0.860). Finally, the SHapley Additive exPlanations (SHAP) analysis is employed to interpret the correlation between the risk features and the outcome.

Figure 1

The framework of stacked ensemble model. In Figure 1(a), five groups of predictions based on 5-fold cross validation are used as the new training set. In Figure 1(b), logistic regression is used for modeling the new training set.

Figure 2

The framework of features and approaches in the experiment. The blue rectangle icons represent feature sets, and the orange ellipse icons represent approaches.

Figure 3

Model performance on test data. (a) Compared with three models, the proposed SEM is superior to others, which has the AUC of 0.879 and 95% CI (0.842,0.883). (b) The performance of SEM based on SetS achieves better prediction.

Figure 4

The rank of feature importance based on the SHAP contribution values. Figure 4 shows the top 10 important features based on the Shapely contribution values. The feature AHDP and AIDP have a strong relationship with the mortality rate.

Figure 5

SHAP interaction analysis plots of the selected features. Abbreviation: ap3diag: apache_3j_diagnosis. (a) The interaction analysis for age and its corresponding SHAP value. (b) The interaction analysis for ap3diag and its corresponding SHAP value. (c) The interaction analysis for d1_bun_min and its corresponding SHAP value. (d) The interaction analysis for gcs_motor_apache and its corresponding SHAP value. (e) The interaction analysis for d1_heartrate_min and its corresponding SHAP value. (f) The interaction analysis for d1_spo2_min and its corresponding SHAP value.

Algorithm 1

APACHE score-based model using stacked ensemble technique.