A Novel Composite Indicator of Predicting Mortality Risk for Heart Failure Patients With Diabetes Admitted to Intensive Care Unit Based on Machine Learning

Abstract

Background: Patients with heart failure (HF) with diabetes may face a poorer prognosis and higher mortality than patients with either disease alone, especially for those in intensive care unit. So far, there is no precise mortality risk prediction indicator for this kind of patient.

Method: Two high-quality critically ill databases, the Medical Information Mart for Intensive Care IV (MIMIC-IV) database and the Telehealth Intensive Care Unit (eICU) Collaborative Research Database (eICU-CRD) Collaborative Research Database, were used for study participants' screening as well as internal and external validation. Nine machine learning models were compared, and the best one was selected to define indicators associated with hospital mortality for patients with HF with diabetes. Existing attributes most related to hospital mortality were identified using a visualization method developed for machine learning, namely, Shapley Additive Explanations (SHAP) method. A new composite indicator ASL was established using logistics regression for patients with HF with diabetes based on major existing indicators. Then, the new index was compared with existing indicators to confirm its discrimination ability and clinical value using the receiver operating characteristic (ROC) curve, decision curve, and calibration curve.

Results: The random forest model outperformed among nine models with the area under the ROC curve (AUC) = 0.92 after hyper-parameter optimization. By using this model, the top 20 attributes associated with hospital mortality in these patients were identified among all the attributes based on SHAP method. Acute Physiology Score (APS) III, Sepsis-related Organ Failure Assessment (SOFA), and Max lactate were selected as major attributes related to mortality risk, and a new composite indicator was developed by combining these three indicators, which was named as ASL. Both in the initial and external cohort, the new indicator, ASL, had greater risk discrimination ability with AUC higher than 0.80 in both low- and high-risk groups compared with existing attributes. The decision curve and calibration curve indicated that this indicator also had a respectable clinical value compared with APS III and SOFA. In addition, this indicator had a good risk stratification ability when the patients were divided into three risk levels.

Conclusion: A new composite indicator for predicting mortality risk in patients with HF with diabetes admitted to intensive care unit was developed on the basis of attributes identified by the random forest model. Compared with existing attributes such as APS III and SOFA, the new indicator had better discrimination ability and clinical value, which had potential value in reducing the mortality risk of these patients.

Keywords: diabetes; heart failure; hospital mortality; indicator; machine learning.

Figure 1

Flowchart of this study.

Figure 2

(A) Cluster diagram in MIMIC-IV population. (B) Survival curve between two clusters of patients in MIMIC-IV population.

Figure 3

(A) Receiver operating characteristic (ROC) curves of the nine models. (B) Precision-Recall (P-R) curves of the nine models.

Figure 4

(A) Receiver operating characteristic (ROC) curves of the Random Forest model after hyper-parameter optimization. (B) Confusion matrix of the Random Forest model after hyper-parameter optimization.

Figure 5

Bar charts that rank the importance of 20 indicators identified by Shapley Additive Explanations (SHAP) values. (A) The overall MIMIC-IV population. (B) Low-risk group. (C) High-risk group.

Figure 6

Distribution of the impact each feature had on the full model output using Shapley Additive Explanations (SHAP) values. (A) The overall MIMIC-IV population. (B) Low-risk group. (C) High-risk group.

Figure 7

Receiver operating characteristic (ROC) curves of three different indicators in MIMIC-IV cohort. (A) The overall MIMIC-IV population. (B) Low-risk group. (C) High-risk group.

Figure 8

(A) Receiver operating characteristic (ROC) curves of three different indicators in eICU cohort. (B) DCA curves of three different indicators in eICU cohort. (C) Calibration curves of three different indicators in eICU cohort. (D) Association between ASL and hospital mortality in MIMIC-IV cohort using Lowess. (E) Association between ASL and hospital mortality in eICU cohort using Lowess.