A Retrospective Cohort Study: Predicting 90-Day Mortality for ICU Trauma Patients with a Machine Learning Algorithm Using XGBoost Using MIMIC-III Database

Abstract

Objective: The aim of this study was to develop and validate a machine learning-based predictive model that predicts 90-day mortality in ICU trauma patients.

Methods: Data of patients with severe trauma were extracted from the Medical Information Mart for Intensive Care III (MIMIC-III) database. The performances of mortality prediction models generated using nine machine learning extreme gradient boosting (XGBoost), logistic regression, random forest, AdaBoost, multilayer perceptron (MLP) neural networks, support vector machine (SVM), light gradient boosting machine (GBM), k nearest neighbors (KNN) and gaussian naive bayes (GNB). The performance of the model was evaluated in terms of discrimination, calibration and clinical application.

Results: We found that the accuracy, sensitivity, specificity, PPV, NPV and F1 score of our proposed XGBoost model were 82.8%, 79.7%, 77.6%, 51.2%, 91.5% and 0.624, respectively. Among the nine models, the XGBoost model performed best. Compared with traditional logistic regression, the calibration curves of the XGBoost model and decision curve analysis (DCA) performed well.

Conclusion: Our study shows that the XGBoost model outperforms other machine learning models in predicting 90-day mortality in trauma patients. It can be used to assist clinicians in the early identification of mortality risk factors and early intervention to reduce mortality.

Keywords: MIMIC-III; XGBoost; intensive care unit; mortality; prediction model; severe trauma patient.

Figure 1

The flow chart of data extraction.

Figure 2

Receiver operating characteristic curves (ROCs) of the XGBoost, logistic regression, random forest, AdaBoost, MLP, GNB, SVM, KNN and LightGBM models. (a) Training sets. (b) Validation sets.

Figure 3

(a) Calibration plots and (b) decision curve analysis (DCA) of the XGBoost model and conventional logistic regression prediction models. The XGBoost calibration curves performed well, and the XGBoost model had a greater net benefit in DCA than the logistic regression model.

Figure 4

Model evaluation and validation in the training and validation sets. Receiver operating characteristic curves (ROCs) of XGBoost. (a) Training sets. (b) Test sets. (c) Learning curve.

Figure 5

Feature importance estimated using the Shapley Additive explanations (SHAP) values. The plot sorts features by the sum of SHAP value magnitudes over all samples. The color represents the feature value (red high, blue low). The x-axis measures the impact on the model output (right positive, left negative). Taking the feature SAPS II as an example, red points are on the right, whereas blue points are on the left. This means prediction scores will be smaller when patients have a lower SAPS II score.

Figure 6

The predicted results for three specific instances. The red and blue bars represent risk factors and protective factors, respectively. Longer lines indicate that the eigenvalues are more important. This figure shows the explanation for a low-risk instance (a), a medium-risk instance (b) and a high-risk instance (c).