Acute myocardial infarction prognosis prediction with reliable and interpretable artificial intelligence system

Abstract

Objective: Predicting mortality after acute myocardial infarction (AMI) is crucial for timely prescription and treatment of AMI patients, but there are no appropriate AI systems for clinicians. Our primary goal is to develop a reliable and interpretable AI system and provide some valuable insights regarding short, and long-term mortality.

Materials and methods: We propose the RIAS framework, an end-to-end framework that is designed with reliability and interpretability at its core and automatically optimizes the given model. Using RIAS, clinicians get accurate and reliable predictions which can be used as likelihood, with global and local explanations, and "what if" scenarios to achieve desired outcomes as well.

Results: We apply RIAS to AMI prognosis prediction data which comes from the Korean Acute Myocardial Infarction Registry. We compared FT-Transformer with XGBoost and MLP and found that FT-Transformer has superiority in sensitivity and comparable performance in AUROC and F1 score to XGBoost. Furthermore, RIAS reveals the significance of statin-based medications, beta-blockers, and age on mortality regardless of time period. Lastly, we showcase reliable and interpretable results of RIAS with local explanations and counterfactual examples for several realistic scenarios.

Discussion: RIAS addresses the "black-box" issue in AI by providing both global and local explanations based on SHAP values and reliable predictions, interpretable as actual likelihoods. The system's "what if" counterfactual explanations enable clinicians to simulate patient-specific scenarios under various conditions, enhancing its practical utility.

Conclusion: The proposed framework provides reliable and interpretable predictions along with counterfactual examples.

Keywords: acute myocardial infarction; counterfactual explanation; electronic health records (EHR); explainable artificial intelligence (XAI); interpretable machine learning.

Figure 1.

Overview of reliable and interpretable artificial intelligence system framework.

Figure 2.

Reliability diagram for each task. The diagram illustrates how well the predicted likelihoods (confidences) of a model correspond to the actual outcomes. In the diagram, predictions are grouped into bins based on their predicted probability. If the model is perfectly mirroring the actual likelihood then the diagram should plot the identity function. Any deviation from a perfect diagonal (red bars) represents discrepancy. These discrepancies are measured using expected calibration error (ECE, the details regarding ECE are in supplementary Appendix). The closer to zero, the better ECE.

Figure 3.

Global Explanations based on SHAP. (A) In-hospital mortality (B) 6-month mortality (C) 12-month mortality. (D–F) represent the F1 scores for recursive feature elimination for in-hospital mortality, 6-month mortality, and 12-month mortality, respectively. The shaded regions indicate the standard deviation over 5 random seeds. (G) Comparison of important features over different periods. The purple line indicates that the feature is included in the top 10 important features of both tasks. The blue and red lines indicate that the feature is included in the top 10 features of one task, but not in the top 10 important features of other tasks.

Figure 4.

Global explanations of the top 5 most important features for each task. The higher the SHAP value, the greater the contribution to mortality, and each dot represents the individual patient.

Figure 5.

Counterfactual examples with local explanations. (A) The change in mortality without and with beta-blocker prescription when left ventricular ejection fraction is less than 40% (B) The change in mortality according to the absence and presence of clopidogrel, one of the platelet aggregation inhibitors. The bold numbers are the predicted likelihood (f(x)), while the base value is the expectation of the training cohort. Features are represented by arrows that push results to high mortality (right arrows) or low mortality (left arrows). The length of the arrows is proportional to the SHAP values of the relevant features for each prediction. Less important features are omitted for visualization.