Machine learning to predict adverse outcomes after cardiac surgery: A systematic review and meta-analysis

Abstract

Background: Machine learning (ML) models are promising tools for predicting adverse postoperative outcomes in cardiac surgery, yet have not translated to routine clinical use. We conducted a systematic review and meta-analysis to assess the predictive performance of ML approaches.

Methods: We conducted an electronic search to find studies assessing ML and traditional statistical models to predict postoperative outcomes. Our primary outcome was the concordance (C-) index of discriminative performance. Using a Bayesian meta-analytic approach we pooled the C-indices with the 95% credible interval (CrI) across multiple outcomes comparing ML methods to logistic regression (LR) and clinical scoring tools. Additionally, we performed critical difference and sensitivity analysis.

Results: We identified 2792 references from the search of which 51 met inclusion criteria. Two postoperative outcomes were amenable for meta-analysis: 30-day mortality and in-hospital mortality. For 30-day mortality, the pooled C-index and 95% CrI were 0.82 (0.79-0.85), 0.80 (0.77-0.84), 0.78 (0.74-0.82) for ML models, LR, and scoring tools respectively. For in-hospital mortality, the pooled C-index was 0.81 (0.78-0.84) and 0.79 (0.73-0.84) for ML models and LR, respectively. There were no statistically significant results indicating ML superiority over LR.

Conclusion: In cardiac surgery patients, for the prediction of mortality, current ML methods do not have greater discriminative power over LR as measured by the C-index.

Keywords: artificial intelligence; cardiac surgery; machine learning; meta-analysis; perioperative risk; systematic review.

Figure 1

Preferred reporting items for systematic reviews and meta‐analysis (PRISMA) flow diagram. The flow diagram shows the flow of studies and exclusions through the different phases of the systematic review.

Figure 2

Distribution of publication rate over time for included studies. The rate of publication of applied machine learning papers in cardiac surgery is currently following an exponential distribution.

Figure 3

Forest plot for 30‐day mortality across model type. Panel (A) shows the subgroup for the best performing machine learning model. Panel (B) shows the subgroup for logistic regression models. Panel (C) shows the subgroup for clinical scoring tools.

Figure 4

Forest plot for in‐hospital mortality across model type. Panel (A) shows the subgroup for the best performing machine learning model. Panel (B) shows the subgroup for logistic regression models.

Figure 5

Critical difference diagram for low‐risk studies. This figure compares the pairwise performance of machine learning (ML) models, logistic regression, and scoring tools and is agnostic to outcome. Groups further to the right are more performant, however, a solid line between groups indicates a lack of statistical significance. In this plot, although there is a trend toward ML models outperforming logistic regression, it is not statistically significant. All other pairwise comparisons are statistically significant.