A Machine-Learning Approach for Dynamic Prediction of Sepsis-Induced Coagulopathy in Critically Ill Patients With Sepsis

Abstract

Background: Sepsis-induced coagulopathy (SIC) denotes an increased mortality rate and poorer prognosis in septic patients. Objectives: Our study aimed to develop and validate machine-learning models to dynamically predict the risk of SIC in critically ill patients with sepsis. Methods: Machine-learning models were developed and validated based on two public databases named Medical Information Mart for Intensive Care (MIMIC)-IV and the eICU Collaborative Research Database (eICU-CRD). Dynamic prediction of SIC involved an evaluation of the risk of SIC each day after the diagnosis of sepsis using 15 predictive models. The best model was selected based on its accuracy and area under the receiver operating characteristic curve (AUC), followed by fine-grained hyperparameter adjustment using the Bayesian Optimization Algorithm. A compact model was developed, based on 15 features selected according to their importance and clinical availability. These two models were compared with Logistic Regression and SIC scores in terms of SIC prediction. Results: Of 11,362 patients in MIMIC-IV included in the final cohort, a total of 6,744 (59%) patients developed SIC during sepsis. The model named Categorical Boosting (CatBoost) had the greatest AUC in our study (0.869; 95% CI: 0.850-0.886). Coagulation profile and renal function indicators were the most important features for predicting SIC. A compact model was developed with an AUC of 0.854 (95% CI: 0.832-0.872), while the AUCs of Logistic Regression and SIC scores were 0.746 (95% CI: 0.735-0.755) and 0.709 (95% CI: 0.687-0.733), respectively. A cohort of 35,252 septic patients in eICU-CRD was analyzed. The AUCs of the full and the compact models in the external validation were 0.842 (95% CI: 0.837-0.846) and 0.803 (95% CI: 0.798-0.809), respectively, which were still larger than those of Logistic Regression (0.660; 95% CI: 0.653-0.667) and SIC scores (0.752; 95% CI: 0.747-0.757). Prediction results were illustrated by SHapley Additive exPlanations (SHAP) values, which made our models clinically interpretable. Conclusions: We developed two models which were able to dynamically predict the risk of SIC in septic patients better than conventional Logistic Regression and SIC scores.

Keywords: Logistic Regression; dynamic prediction; external validation; machine learning; model interpretation; sepsis-induced coagulopathy.

Figure 1

Schematic illustration of the study design. (A) Design of dynamic prediction in our study. Daily assessment was performed from the time when sepsis was diagnosed. If SIC criteria were not fulfilled, the risk of SIC the next day was predicted by our model. Prediction stopped when SIC was diagnosed, and restarted when patients recovered from SIC. (B) Schematic illustration of model development. We compared the discrimination of 15 machine-learning models using 10-fold cross-validation. The model with the best accuracy and greatest AUC was chosen. Fine-grained hyperparameter adjustment was performed using Bayesian Optimization. Fifteen features were selected according to their SHAP values and clinical availability. A compact model was developed based on the selected features. Lastly, these two models were validated in eICU-CRD. ICU, intensive care unit; SIC, sepsis-induced coagulopathy; SHAP, SHapley Additive exPlanations; MIMIC-IV, Medical Information Mart for Intensive Care-IV; C.V., cross-validation; eICU-CRD, the eICU Collaborative Research Database.

Figure 2

Flow chart of patient selection.

Figure 3

Distribution of the impact each feature had on the full model output estimated using the SHapley Additive exPlanations (SHAP) values. The plot sorts features by the sum of SHAP value magnitudes over all samples. The color represents the feature value (red high, blue low). The x axis measures the impact on the model output (right positive, left negative). Taking the feature platelet as an example, red points are on the left whereas blue points are on the right. This means prediction scores will be smaller when patients have a low level of platelets. PT, prothrombin time; INR, international normalized ratio; SIC, sepsis-induced coagulopathy; SIC platelet, platelet term in the SIC score; SOFA, sequential organ failure assessment; PTT, Partial Thromboplastin Time; BMI, body mass index; MAP, mean arterial pressure; WBC, white blood cell count; RDW, red cell distribution width; MV, mechanical ventilation.

Figure 4

AUCs of four predictive methods in internal (MIMIC-IV) and external (eICU-CRD) validations. AUCs of our two models, Logistic Regression and SIC scores were assessed using the Bootstrap Resampling technique with 1,000 iterations. The heights of the bars represent the median AUCs, while the error bars represent the 95% confidence intervals. Full, the full model; Comp, the compact model; LR, Logistic Regression; SIC, the sepsis-induced coagulopathy criteria; AUC, area under receiver operating characteristic curve; MIMIC-IV, Medical Information Mart for Intensive Care-IV; eICU-CRD, the eICU Collaborative Research Database.

Figure 5

Model performance in different patient cohorts in eICU-CRD. Different validation sets were derived based on APACHE-IV (A), age (B), region of the United States (C), ethnicity (D), time since sepsis onset (E) and unit type (F). AUC of the full and the compact models in each set was measured using the Bootstrap Resampling technique. The colored area represents 95% confidence intervals. Full, the full model; Comp, the compact model; AUC, area under receiver operating characteristic curve; APACHE-IV, Acute Physiology and Chronic Health Evaluation-IV; CICU, cardiac intensive care unit; CSICU, cardiac surgical intensive care unit; CTICU, cardiothoracic intensive care unit; MICU, medical intensive care unit; NICU, neuro intensive care unit; SICU, surgical intensive care unit.

Figure 6

Explanation of the prediction results for specific instances. The base value (−3.33) is the average value of the predictive model; the output values are the predicted SIC risks. The bars in red and blue represent risk factors and protective factors, respectively; longer bars mean greater feature importance. Here, these values are the model outputs before the SoftMax layer, and therefore, they are not equal to the final predicted probabilities. This figure shows the explanation for a high-risk instance (A) and a low-risk instance (B). RDW, red cell distribution width; PT, prothrombin time; WBC, white blood cell count; PTT, Partial Thromboplastin Time; INR, international normalized ratio; MAP, mean arterial pressure; BMI, body mass index.