A hybrid modeling framework for generalizable and interpretable predictions of ICU mortality across multiple hospitals

Abstract

The development of reliable mortality risk stratification models is an active research area in computational healthcare. Mortality risk stratification provides a standard to assist physicians in evaluating a patient's condition or prognosis objectively. Particular interest lies in methods that are transparent to clinical interpretation and that retain predictive power once validated across diverse datasets they were not trained on. This study addresses the challenge of consolidating numerous ICD codes for predictive modeling of ICU mortality, employing a hybrid modeling approach that integrates mechanistic, clinical knowledge with mathematical and machine learning models . A tree-structured network connecting independent modules that carry clinical meaning is implemented for interpretability. Our training strategy utilizes graph-theoretic methods for data analysis, aiming to identify the functions of individual black-box modules within the tree-structured network by harnessing solutions from specific max-cut problems. The trained model is then validated on external datasets from different hospitals, demonstrating successful generalization capabilities, particularly in binary-feature datasets where label assessment involves extrapolation.

Keywords: Generalizability; Hybrid modeling; ICD codes; ICU mortality prediction; Interpretability; Machine learning.

Figure 1

Average and the 95% confidence interval of the Jaccard similarity measures between data samples from a validation hospital and the Derivation Hospital, emphasizing the degree of relatedness between the Derivation Hospital and four validation hospitals.

Figure 2

Proposed structured hybrid model for mortality risk stratification of critically ill, influenza and pneumonia patients in the ICU. The model consists of five modules: kidney failure, infectious and bacterial diseases, liver failure, mental and psychic, and lung failure; with their corresponding input features. The output module combines the precomputations of these modules to determine the overall mortality risk of a patient.

Figure 3

AUC–ROC curves comparing the discriminative ability of our hybrid model and the XGBoost model in distinguishing deceased and alive patients. The hybrid model outperformed XGBoost for Validation hospitals 1, 3, and 4, where their similarities with the Derivation Hospital are less pronounced, highlighting the hybrid model’s generalizability.

Figure 4

SHAP values distribution for 12 ICD codes in the XGBoost model, used to interpret ICU mortality causes. The figure showcases inconsistency in the feature importance across the five hospitals involved in the study.

Figure 5

SHAP value distribution for the hybrid model’s black-box modules across five hospitals. The consistency across hospitals showcases the hybrid model’s interpretability, reliability, and stability in mortality prediction across diverse healthcare settings.

Figure 6

Simple case: a tree-structured network with three first-layer modules mapping 7-dimensional binary input variable to binary outputs.

Figure 7

The schematic representation of $T_0$ for the simple case, which contains $2^7$ elements holding the number of 0 labels for each input configuration in given training data.

Algorithm 1

Risk stratification algorithm.

Figure 8

(a) All $2^7$ possible binary inputs of ${\mathbf{F}}_{\mathit{simple}}$. Each row runs along 8 input configuration $V_1=\{1,2,\dots ,8\}$ of Module-1 and depicts the inputs variables of ${\mathbf{F}}_{\mathit{simple}}$ with fixed inputs to Module-2 and Module-3. The blue cells in the same row depict all 16 possible pairs of input variables for which the decimal representation of the inputs to the 3 first-layer modules are like (1, j, k) and (4, j, k). (b) To determine the weights of the conflict graph $G_1(V_1,E_1)$ of Module-1, we compare the labels of input variables within the same row. (c) The conflict graph $G_1(V_1,E_1)$ of Module-1 with both binary and decimal representations of vertices. In the risk stratification algorithm, the value of edge $w_{14}$ results from Eq. (4) iterated over all $j\in V_2=\{1,2,3,4\}$ and $k\in V_3=\{1,2,3,4\}$.