Prediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores

Abstract

Objective: Current cardiac surgery risk models do not address a substantial fraction of procedures. We sought to create models to predict the risk of operative mortality for an expanded set of cases.

Methods: Four supervised machine learning models were trained using preoperative variables present in the Society of Thoracic Surgeons (STS) data set of the Massachusetts General Hospital to predict and classify operative mortality in procedures without STS risk scores. A total of 424 (5.5%) mortality events occurred out of 7745 cases. Models included logistic regression with elastic net regularization (LogReg), support vector machine, random forest (RF), and extreme gradient boosted trees (XGBoost). Model discrimination was assessed via area under the receiver operating characteristic curve (AUC), and calibration was assessed via calibration slope and expected-to-observed event ratio. External validation was performed using STS data sets from Brigham and Women's Hospital (BWH) and the Johns Hopkins Hospital (JHH).

Results: Models performed comparably with the highest mean AUC of 0.83 (RF) and expected-to-observed event ratio of 1.00. On external validation, the AUC was 0.81 in BWH (RF) and 0.79 in JHH (LogReg/RF). Models trained and applied on the same institution's data achieved AUCs of 0.81 (BWH: LogReg/RF/XGBoost) and 0.82 (JHH: LogReg/RF/XGBoost).

Conclusions: Machine learning models trained on preoperative patient data can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. Such procedures comprise 23% of all cardiac surgical procedures nationwide. This work also highlights the value of using local institutional data to train new prediction models that account for institution-specific practices.

Keywords: cardiac surgery; machine learning; operative mortality; risk prediction.

FIGURE 1.

Receiver operating characteristic (A) and calibration curves (B) of mortality risk models for cardiac surgical procedures without Society of Thoracic Surgeons risk scores at the Massachusetts General Hospital using preoperative variables. AUC, Area under the receiver operating characteristic curve; CI, confidence interval.

FIGURE 2.

Machine learning can accurately predict mortality for procedures without STS scores.

FIGURE 3.

Parsimonious model trained on subset of preoperative variables from the Massachusetts General Hospital (MGH; A; top variables, receiver operating characteristic and calibration curves shown). MGH-trained model applied to the Brigham and Women’s Hospital (BWH) cohort (B) and the Johns Hopkins Hospital (JHH) cohort (C); receiver operating characteristic and calibration curves shown. AUC, Area under the receiver operating characteristic curve; CI, confidence interval.

FIGURE 4.

Machine learning (ML) models can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. MGH, Massachusetts General Hospital; BWH, Brigham and Women’s Hospital; AUC, area under the receiver operating characteristic curve; XGBoost, extreme gradient boosted trees.