Explainable Artificial Intelligence Warning Model Using an Ensemble Approach for In-Hospital Cardiac Arrest Prediction: Retrospective Cohort Study

Abstract

Background: Cardiac arrest (CA) is the leading cause of death in critically ill patients. Clinical research has shown that early identification of CA reduces mortality. Algorithms capable of predicting CA with high sensitivity have been developed using multivariate time series data. However, these algorithms suffer from a high rate of false alarms, and their results are not clinically interpretable.

Objective: We propose an ensemble approach using multiresolution statistical features and cosine similarity-based features for the timely prediction of CA. Furthermore, this approach provides clinically interpretable results that can be adopted by clinicians.

Methods: Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV database and the eICU Collaborative Research Database. Based on the multivariate vital signs of a 24-hour time window for adults diagnosed with heart failure, we extracted multiresolution statistical and cosine similarity-based features. These features were used to construct and develop gradient boosting decision trees. Therefore, we adopted cost-sensitive learning as a solution. Then, 10-fold cross-validation was performed to check the consistency of the model performance, and the Shapley additive explanation algorithm was used to capture the overall interpretability of the proposed model. Next, external validation using the eICU Collaborative Research Database was performed to check the generalization ability.

Results: The proposed method yielded an overall area under the receiver operating characteristic curve (AUROC) of 0.86 and area under the precision-recall curve (AUPRC) of 0.58. In terms of the timely prediction of CA, the proposed model achieved an AUROC above 0.80 for predicting CA events up to 6 hours in advance. The proposed method simultaneously improved precision and sensitivity to increase the AUPRC, which reduced the number of false alarms while maintaining high sensitivity. This result indicates that the predictive performance of the proposed model is superior to the performances of the models reported in previous studies. Next, we demonstrated the effect of feature importance on the clinical interpretability of the proposed method and inferred the effect between the non-CA and CA groups. Finally, external validation was performed using the eICU Collaborative Research Database, and an AUROC of 0.74 and AUPRC of 0.44 were obtained in a general intensive care unit population.

Conclusions: The proposed framework can provide clinicians with more accurate CA prediction results and reduce false alarm rates through internal and external validation. In addition, clinically interpretable prediction results can facilitate clinician understanding. Furthermore, the similarity of vital sign changes can provide insights into temporal pattern changes in CA prediction in patients with heart failure-related diagnoses. Therefore, our system is sufficiently feasible for routine clinical use. In addition, regarding the proposed CA prediction system, a clinically mature application has been developed and verified in the future digital health field.

Keywords: cardiac arrest prediction; cost-sensitive learning; electronic medical records; ensemble learning; temporal pattern changes.

Figure 1

Overview of the proposed CA prediction framework. CA: cardiac arrest; DBP: diastolic blood pressure; HR: heart rate; MBP: mean blood pressure; MEWS: modified early warning score; MIMIC: Medical Information Mart for Intensive Care; RR: respiratory rate; SBP: systolic blood pressure; SpO₂: oxyhemoglobin saturation; TEMP: temperature.

Figure 2

Patient inclusion and exclusion flow diagram for the Medical Information Mart for Intensive Care-IV database. CA: cardiac arrest; ICU: intensive care unit; n: number of stays; SAPS: simplified acute physiology score; SOFA: sequential organ failure assessment.

Figure 3

Comparison among baseline models and the proposed method using a 24-hour time window from the eICU Collaborative Research Database. (A) AUROC; (B) AUPRC. The baseline and proposed models were trained on the Medical Information Mart for Intensive Care-IV database. After the training procedure, we validated the baseline models and the proposed model to estimate generalization ability. We have presented 95% CIs after 1000 bootstrap iterations. AUPRC: area under the precision-recall curve; AUROC: area under the receiver operating characteristic curve; DT: decision tree; GB: Gaussian naïve Bayes; KNN: k-nearest neighbors; LGB: gradient boosting ensemble of decision trees; LR: logistic regression; MLP: multilayer perceptron; RF: random forest; SVM: support vector machine; XGB: extreme gradient boosting ensemble of decision trees.

Figure 4

Clinical interpretability results. (A) Global feature impact values produced by the proposed model. (B) Cosine similarity feature set between the non-CA and CA groups. (C) Multiresolution statistical features based on the cosine similarity matrix between the non-CA and CA groups. (D) Statistical feature set between the non-CA and CA groups. C: channel-level average; CA: cardiac arrest; Cos: cosine similarity; DBP: diastolic blood pressure; HR: heart rate; MBP: mean blood pressure; MEWS: modified early warning score; MIMIC: Medical Information Mart for Intensive Care; RR: respiratory rate; SBP: systolic blood pressure; SHAP: Shapley additive explanation; SpO2: oxyhemoglobin saturation; TEMP: temperature; W: weighted matrix.

Figure 5

Comparison of AUROC values achieved by the proposed model and a state-of-the-art model. The light green line indicates the proposed model, while the blue line represents the method proposed by Layeghian Javan et al [16]. AUROC: area under the receiver operating characteristic curve.