Interpretable machine learning for 28-day all-cause in-hospital mortality prediction in critically ill patients with heart failure combined with hypertension: A retrospective cohort study based on medical information mart for intensive care database-IV and eICU databases

Abstract

Background: Heart failure (HF) combined with hypertension is an extremely important cause of in-hospital mortality, especially for the intensive care unit (ICU) patients. However, under intense working pressure, the medical staff are easily overwhelmed by the large number of clinical signals generated in the ICU, which may lead to treatment delay, sub-optimal care, or even wrong clinical decisions. Individual risk stratification is an essential strategy for managing ICU patients with HF combined with hypertension. Artificial intelligence, especially machine learning (ML), can develop superior models to predict the prognosis of these patients. This study aimed to develop a machine learning method to predict the 28-day mortality for ICU patients with HF combined with hypertension.

Methods: We enrolled all critically ill patients with HF combined with hypertension in the Medical Information Mart for IntensiveCare Database-IV (MIMIC-IV, v.1.4) and the eICU Collaborative Research Database (eICU-CRD) from 2008 to 2019. Subsequently, MIMIC-IV was divided into training cohort and testing cohort in an 8:2 ratio, and eICU-CRD was designated as the external validation cohort. The least absolute shrinkage and selection operator (LASSO) Cox regression with internal tenfold cross-validation was used for data dimension reduction and identifying the most valuable predictive features for 28-day mortality. Based on its accuracy and area under the curve (AUC), the best model in the validation cohort was selected. In addition, we utilized the Shapley Additive Explanations (SHAP) method to highlight the importance of model features, analyze the impact of individual features on model output, and visualize an individual's Shapley values.

Results: A total of 3,458 and 6582 patients with HF combined with hypertension in MIMIC-IV and eICU-CRD were included. The patients, including 1,756 males, had a median (Q1, Q3) age of 75 (65, 84) years. After selection, 22 out of a total of 58 clinical parameters were extracted to develop the machine-learning models. Among four constructed models, the Neural Networks (NN) model performed the best predictive performance with an AUC of 0.764 and 0.674 in the test cohort and external validation cohort, respectively. In addition, a simplified model including seven variables was built based on NN, which also had good predictive performance (AUC: 0.741). Feature importance analysis showed that age, mechanical ventilation (MECHVENT), chloride, bun, anion gap, paraplegia, rdw (RDW), hyperlipidemia, peripheral capillary oxygen saturation (SpO₂), respiratory rate, cerebrovascular disease, heart rate, white blood cell (WBC), international normalized ratio (INR), mean corpuscular hemoglobin concentration (MCHC), glucose, AIDS, mean corpuscular volume (MCV), N-terminal pro-brain natriuretic peptide (Npro. BNP), calcium, renal replacement therapy (RRT), and partial thromboplastin time (PTT) were the top 22 features of the NN model with the greatest impact. Finally, after hyperparameter optimization, SHAP plots were employed to make the NN-based model interpretable with an analytical description of how the constructed model visualizes the prediction of death.

Conclusion: We developed a predictive model to predict the 28-day mortality for ICU patients with HF combined with hypertension, which proved superior to the traditional logistic regression analysis. The SHAP method enables machine learning models to be more interpretable, thereby helping clinicians to better understand the reasoning behind the outcome and assess in-hospital outcomes for critically ill patients.

Keywords: MIMIC-IV; heart failure; hypertension; interpretable machine learning; neural networks.

FIGURE 1

A flow chart describing the procedure for subject selection.

FIGURE 2

Demographic and clinical feature selection using the least absolute shrinkage and selection operator (LASSO) binary logistic regression model. (A) Tuning parameter (λ) selection in the LASSO model used 10- fold cross-validation via minimum criteria. The partial likelihood deviance (binomial deviance) curve was plotted versus log(λ). Dotted vertical lines were drawn at the optimal values by using the minimum criteria and the one SE of the minimum criteria (the 1-SE criteria). λ value of 0.01914052, with log(λ), −3.9545 was chosen (1- SE criteria) according to 10-fold cross-validation. (B) LASSO coefficient profiles of the 58 features. A coefficient profile plot was produced against the log(λ) sequence. The vertical line was drawn at the value selected using 10-fold cross-validation, where optimal resulted in 20 features with non-zero coefficients.

FIGURE 3

SHAP summary plot for the top 22 clinical features contributing to the NN (A) and LR (B) model. SHAP feature importance is measured as one minus AUC loss after permutations. This matrix plot depicts the importance of each covariate in the development of the final predictive model.

FIGURE 4

The receiver operator characteristic (ROC) curves for the ML models predict in-hospital mortality (The training cohort and testing cohort). ROC curves for five ML models predicting in-hospital mortality in the training (A) and testing cohort (B), respectively; (C) ROC curves for in-hospital mortality in the test set predicted by the un-simplified NN model and the ROC curves for in-hospital mortality in the test set and external validation set predicted by the simplified NN model, respectively.

FIGURE 5

SHAP contribution values of different variables for a single sample of the NN model.