Early prognosis prediction for non-variceal upper gastrointestinal bleeding in the intensive care unit: based on interpretable machine learning

Abstract

Introduction: This study aims to construct a mortality prediction model for patients with non-variceal upper gastrointestinal bleeding (NVUGIB) in the intensive care unit (ICU), employing advanced machine learning algorithms. The goal is to identify high-risk populations early, contributing to a deeper understanding of patients with NVUGIB in the ICU.

Methods: We extracted NVUGIB data from the Medical Information Mart for Intensive Care IV (MIMIC-IV, v.2.2) database spanning from 2008 to 2019. Feature selection was conducted through LASSO regression, followed by training models using 11 machine learning methods. The best model was chosen based on the area under the curve (AUC). Subsequently, Shapley additive explanations (SHAP) was employed to elucidate how each factor influenced the model. Finally, a case was randomly selected, and the model was utilized to predict its mortality, demonstrating the practical application of the developed model.

Results: In total, 2716 patients with NVUGIB were deemed eligible for participation. Following selection, 30 out of a total of 64 clinical parameters collected on day 1 after ICU admission remained associated with prognosis and were utilized for developing machine learning models. Among the 11 constructed models, the Gradient Boosting Decision Tree (GBDT) model demonstrated the best performance, achieving an AUC of 0.853 and an accuracy of 0.839 in the validation cohort. Feature importance analysis highlighted that shock, Glasgow Coma Scale (GCS), renal disease, age, albumin, and alanine aminotransferase (ALP) were the top six features of the GBDT model with the most significant impact. Furthermore, SHAP force analysis illustrated how the constructed model visualized the individualized prediction of death.

Conclusions: Patient data from the MIMIC database were leveraged to develop a robust prognostic model for patients with NVUGIB in the ICU. The analysis using SHAP also assisted clinicians in gaining a deeper understanding of the disease.

Fig. 1

Flowchart of this study

Fig. 2

Receiver operator characteristic (ROC) curves for the ML models and the traditional severity of illness scores to predict in-hospital mortality (validation cohort). A ROC curves for the seven ML models to predict in-hospital mortality; B ROC curves for the traditional severity of illness scores to predict in-hospital mortality. Support Vector Machine (SVM), k-Nearest Neighbors (KNN), eXtreme Gradient Boosting (XGBoost), Decision Tree (DT), Naive Bayes (NB), Random Forest (RF), Adaptive Boosting (AdaBoost), Gradient Boosting Decision Tree (GBDT), Multilayer Perceptron (MLP), Light Gradient Boosting Machine (LGBM), Catboost (Cat), and logistic regression (LR), Glasgow Blatchford Score (GBS) and AIMS65 score (AIM)

Fig. 3

SHAP summary plot for the top 20 clinical features contributing to the GBDT model. A SHAP feature importance measured as the mean absolute Shapley values. This matrix plot depicts the importance of each covariate in the development of the final predictive model. B The attributes of the features in the model. The position on the y-axis is determined by the feature and on the x-axis by the Shapley value. The color represents the value of the feature from low to high

Fig. 4

SHAP dependency plot for the top 6 clinical features contributing to GBDT model. A GCS, B age, C albumin, D ALP, E respiratory rate, F serum chloride. SHAP values for specific features exceed zero, representing an increased risk of death

Fig. 5

SHAP force plot for explaining of individual’s prediction results in the validation cohort. Screenshot of the death prediction in patients with NVUGIB. A, B Model predictions by randomly drawing a single sample from the validation cohort. Redder sample points indicate that the value of the feature is larger, and bluer sample points indicate that the value of the feature is smaller