Systematic review and meta-analysis of mortality risk prediction models in adult cardiac surgery

Abstract

Objectives: The most used mortality risk prediction models in cardiac surgery are the European System for Cardiac Operative Risk Evaluation (ES) and Society of Thoracic Surgeons (STS) score. There is no agreement on which score should be considered more accurate nor which score should be utilized in each population subgroup. We sought to provide a thorough quantitative assessment of these 2 models.

Methods: We performed a systematic literature review and captured information on discrimination, as quantified by the area under the receiver operator curve (AUC), and calibration, as quantified by the ratio of observed-to-expected mortality (O:E). We performed random effects meta-analysis of the performance of the individual models as well as pairwise comparisons and subgroup analysis by procedure type, time and continent.

Results: The ES2 {AUC 0.783 [95% confidence interval (CI) 0.765-0.800]; O:E 1.102 (95% CI 0.943-1.289)} and STS [AUC 0.757 (95% CI 0.727-0.785); O:E 1.111 (95% CI 0.853-1.447)] showed good overall discrimination and calibration. There was no significant difference in the discrimination of the 2 models (difference in AUC -0.016; 95% CI -0.034 to -0.002; P = 0.09). However, the calibration of ES2 showed significant geographical variations (P < 0.001) and a trend towards miscalibration with time (P=0.057). This was not seen with STS.

Conclusions: ES2 and STS are reliable predictors of short-term mortality following adult cardiac surgery in the populations from which they were derived. STS may have broader applications when comparing outcomes across continents as compared to ES2.

Registration: Prospero (https://www.crd.york.ac.uk/PROSPERO/) CRD42020220983.

Keywords: Cardiac surgery; European System for Cardiac Operative Risk Evaluation; Mortality; Prediction; Society of Thoracic Surgeons.

Figure 1:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart.

Figure 2:

Forest plots of meta-analysis of European System for Cardiac Operative Risk Evaluation 2. (A) Area under the receiver operator curve. (B) Observed-to-expected ratio.

Figure 3:

Forest plots of meta-analysis of Society of Thoracic Surgeons score. (A) Area under the receiver operator curve. (B) Observed-to-expected ratio.

Figure 4:

Difference in discrimination of European System for Cardiac Operative Risk Evaluation 2 and Society of Thoracic Surgeons score. TE: difference in C-stastistic; seTE: standard error of difference in C-statistic.