Comparing the Performance of Machine Learning Models and Conventional Risk Scores for Predicting Major Adverse Cardiovascular Cerebrovascular Events After Percutaneous Coronary Intervention in Patients With Acute Myocardial Infarction: Systematic Review and Meta-Analysis

Abstract

Background: Machine learning (ML) models may offer greater clinical utility than conventional risk scores, such as the Thrombolysis in Myocardial Infarction (TIMI) and Global Registry of Acute Coronary Events (GRACE) risk scores. However, there is a lack of knowledge on whether ML or traditional models are better at predicting the risk of major adverse cardiovascular and cerebrovascular events (MACCEs) in patients with acute myocardial infarction (AMI) who have undergone percutaneous coronary interventions (PCI).

Objective: The aim of this study is to systematically review and critically appraise studies comparing the performance of ML models and conventional risk scores for predicting MACCEs in patients with AMI who have undergone PCI.

Methods: Nine academic and electronic databases including PubMed, CINAHL, Embase, Web of Science, Scopus, ACM, IEEE, Cochrane, and Google Scholar were systematically searched from January 1, 2010, to December 31, 2024. We included studies of patients with AMI who underwent PCI, and predicted MACCE risk using ML algorithms or conventional risk scores. We excluded conference abstracts, gray literature, reviews, case reports, editorials, qualitative studies, secondary data analyses, and non-English publications. Our systematic search yielded 10 retrospective studies, with a total sample size of 89,702 individuals. Three validation tools were used to assess the validity of the published prediction models. Most included studies were assessed as having a low overall risk of bias.

Results: The most frequently used ML algorithms were random forest (n=8) and logistic regression (n=6), while the most used conventional risk scores were GRACE (n=8) and TIMI (n=4). The most common MACCEs component was 1-year mortality (n=3), followed by 30-day mortality (n=2) and in-hospital mortality (n=2). Our meta-analysis demonstrated that ML-based models (area under the receiver operating characteristic curve: 0.88, 95% CI 0.86-0.90; I²=97.8%; P<.001) outperformed conventional risk scores (area under the receiver operating characteristic curve: 0.79, 95% CI 0.75-0.84; I²=99.6%; P<.001) in predicting mortality risk among patients with AMI who underwent PCI. Heterogeneity across studies was high. Publication bias was assessed using a funnel plot. The top-ranked predictors of mortality in both ML and conventional risk scores were age, systolic blood pressure, and Killip class.

Conclusions: This review demonstrated that ML-based models had superior discriminatory performance compared to conventional risk scores for predicting MACCEs in patients with AMI who had undergone PCI. The most commonly used predictors were confined to nonmodifiable clinical characteristics. Therefore, health care professionals should understand the advantages and limitations of ML algorithms and conventional risk scores before applying them in clinical practice. We highlight the importance of incorporating modifiable factors-including psychosocial and behavioral variables-into prediction models for MACCEs following PCI in patients with AMI. In addition, further multicenter prospective studies with external validation are required to address validation limitations.

Keywords: machine learning; mortality; myocardial infarction; patient readmission; percutaneous coronary intervention; prediction algorithm; statistical model.

Figure 1.. PRISMA flow diagram of study selection. AMI: acute myocardial infarction; MACCEs: major adverse cardiac and cerebrovascular events; PCI: percutaneous coronary intervention; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Figure 2.. Forest plot of pooled AUROC estimates from a random-effects meta-analysis: machine learning models. AUROC: area under the receiver operating characteristic curve; DAMI: deep-learning–based risk stratification for the mortality of patients with acute myocardial infarction; ML: machine learning; MLR: multivariate logistic regression; RF: random forest; SVM: support vector machine; SVMvarImp-SBE-SVM: SVM variable importance with sequential backward elimination and SVM classifier.

Figure 3.. Forest plot of pooled AUROC estimates from a random-effects meta-analysis: conventional risk score models. ACTION: Acute Coronary Treatment and Intervention Outcomes Network scores; AUROC: area under the receiver operating characteristic curve; CRS: conventional risk score; GRACE: Global Registry of Acute Cardiac Events; TIMI: Thrombolysis in Myocardial Infarction.