Interpretable machine learning model for early prediction of 28-day mortality in ICU patients with sepsis-induced coagulopathy: development and validation

Abstract

Objective: Sepsis-induced coagulopathy (SIC) is extremely common in individuals with sepsis, significantly associated with poor outcomes. This study attempted to develop an interpretable and generalizable machine learning (ML) model for early predicting the risk of 28-day death in patients with SIC.

Methods: In this retrospective cohort study, we extracted SIC patients from the Medical Information Mart for Intensive Care III (MIMIC-III), MIMIC-IV, and eICU-CRD database according to Toshiaki Iba's scale. And the overlapping in the MIMIC-IV was excluded for this study. Afterward, only the MIMIC-III cohort was randomly divided into the training set, and the internal validation set according to the ratio of 7:3, while the MIMIC-IV and eICU-CRD databases were considered the external validation sets. The predictive factors for 28-day mortality of SIC patients were determined using recursive feature elimination combined with tenfold cross-validation (RFECV). Then, we constructed models using ML algorithms. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUROC), area under the precision recall curve (AUPRC), accuracy, sensitivity, specificity, negative predictive value, positive predictive value, recall, and F1 score. Finally, Shapley Additive Explanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) were employed to provide a reasonable interpretation for the prediction results.

Results: A total of 3280, 2798, and 1668 SIC patients were screened from MIMIC-III, MIMIC-IV, and eICU-CRD databases, respectively. Seventeen features were selected to construct ML prediction models. XGBoost had the best performance in predicting the 28-day mortality of SIC patients, with AUC of 0.828, 0.913 and 0.923, the AUPRC of 0.807, 0.796 and 0.921, the accuracy of 0.785, 0.885 and 0.891, the F₁ scores were 0.63, 0.69 and 0.70 in MIMIC-III (internal validation set), MIMIC-IV, and eICU-CRD databases. The importance ranking and SHAP analyses showed that initial SOFA score, red blood cell distribution width (RDW), and age were the top three critical features in the XGBoost model.

Conclusions: We developed an optimal and explainable ML model to predict the risk of 28-day death of SIC patients 28-day death risk. Compared with conventional scoring systems, the XGBoost model performed better. The model established will have the potential to improve the level of clinical practice for SIC patients.

Keywords: Gradient boosting decision tree; Local interpretable model-agnostic explanations; Machine learning; Sepsis induced coagulopathy; Shapley additive explanations.

Fig. 1

The flowchart and framework of the prediction models

Fig. 2

Receiver operating characteristic curves and area under the precision recall curve showing 28-day death of SIC patients predictive performance of two severity scoring and four machine learning algorithms based on the selected features in the internal validation set (MIMIC-III) (A, D), MIMIC-IV (B, E), and eICU-CRD (C, F) database. LR logistic regression, NG naive bayes, SVM support vector machine, SOFA sequential organ failure assessment, SAPS II simplified acute physiology score II, SIC sepsis-induced coagulopathy, AUC area under the receiver operating characteristic curve

Fig. 3

The interpretation of the XGBoost model. A Feature importance ranking based on SHAP values. The position on the Y-axis implied the importance ranking, and the X-axis reflected the association between each value of features and the corresponding SHAP value. B The importance ranking of included features according to the mean (|SHAP value|). SOFA sequential organ failure assessment, RDW red blood cell distribution width, MCV mean corpuscular volume, BUN blood urea nitrogen, MBP mean blood pressure, WBC white blood cell, MCHC mean corpuscular hemoglobin concentration

Fig. 4

The partial dependence plots of the XGboost model based on SHAP. A-P show how the RDW_max, age, MCV_min, Heartrate_mean, Tempc_mean, Resprate_mean, Po2_min, PT_max, MAP, platelet_min, lactate_max, WBC_max, PTT_max, gender and MCHC_min affects the output of the XGBoost prediction model respectively. As the SHAP value exceeds zero, it indicated a promoting effect on the 28-day death risk. RDW=red blood cell distribution width; MCV=mean corpuscular volume; BUN=blood urea nitrogen; MBP=mean blood pressure; WBC=white blood cell; MCHC=mean corpuscular hemoglobin concentration