Machine-learning versus traditional approaches for atherosclerotic cardiovascular risk prognostication in primary prevention cohorts: a systematic review and meta-analysis

Abstract

Background: Cardiovascular disease (CVD) risk prediction is important for guiding the intensity of therapy in CVD prevention. Whilst current risk prediction algorithms use traditional statistical approaches, machine learning (ML) presents an alternative method that may improve risk prediction accuracy. This systematic review and meta-analysis aimed to investigate whether ML algorithms demonstrate greater performance compared with traditional risk scores in CVD risk prognostication.

Methods and results: MEDLINE, EMBASE, CENTRAL, and SCOPUS Web of Science Core collections were searched for studies comparing ML models to traditional risk scores for CVD risk prediction between the years 2000 and 2021. We included studies that assessed both ML and traditional risk scores in adult (≥18 year old) primary prevention populations. We assessed the risk of bias using the Prediction Model Risk of Bias Assessment Tool (PROBAST) tool. Only studies that provided a measure of discrimination [i.e. C-statistics with 95% confidence intervals (CIs)] were included in the meta-analysis. A total of 16 studies were included in the review and meta-analysis (3302 515 individuals). All study designs were retrospective cohort studies. Out of 16 studies, 3 externally validated their models, and 11 reported calibration metrics. A total of 11 studies demonstrated a high risk of bias. The summary C-statistics (95% CI) of the top-performing ML models and traditional risk scores were 0.773 (95% CI: 0.740-0.806) and 0.759 (95% CI: 0.726-0.792), respectively. The difference in C-statistic was 0.0139 (95% CI: 0.0139-0.140), P < 0.0001.

Conclusion: ML models outperformed traditional risk scores in the discrimination of CVD risk prognostication. Integration of ML algorithms into electronic healthcare systems in primary care could improve identification of patients at high risk of subsequent CVD events and hence increase opportunities for CVD prevention. It is uncertain whether they can be implemented in clinical settings. Future implementation research is needed to examine how ML models may be utilized for primary prevention.This review was registered with PROSPERO (CRD42020220811).

Keywords: Cardiovascular disease risk prediction; Machine learning; Risk prediction algorithms.

Figure 1

PRISMA flow chart (1). Database searches were conducted on Embase, Medline, CENTRAL, and Web of Science on 21 January 2020, and updated searches were conducted on 28 April 2021 and 22 December 2021. These searches identified 7086, 2648, and 2155 unique records, respectively, within these databases using the same search criteria. A total of 10 full-text articles included in the systematic review were identified through database searches and 6 were identified through citation chaining and expert discussion. A total of 16 studies were included in the systematic review, and 15 studies were included in the primary meta-analysis in order to avoid unit-of-error analysis.²⁸

Figure 2

PROBAST risk of bias assessment summary for machine learning models in the included studies.

Figure 3

Primary outcome meta-analysis—top performing machine learning model vs. top perfoming traditional risk scores for the prediction of major adverse cardiovascular events in primary preventiong participant cohorts. FRS, Framingham risk score; ASCVD, American Heart Association Atherosclerotic Cardiovascular Disease; REGICOR, Registre Gironi del Cor; SCORE, systematic coronary risk evaluation; UKPDS, United Kingdom Prospective Diabetes Study; T1DM, type 1 diabetes mellitus; SVM, support vector machine; NN, neural network; RF, random forests; XGBoost, eXtreme gradient boosting; QDA, quadratic discriminant analysis; AutoML, auto machine learning; CoxPH, Cox proportional hazard; ML, machine learning; TRS, traditional risk score.

Figure 4

Externally validated machine learning model vs. traditional risk score. Poisson reg, Poisson regression; SVM, support vector machine; ASCVD, American Heart Association Absolute Cardiovascular Disease Risk Score; ML, machine learning; TRS, traditional risk score.