Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Apr;165(4):1449-1459.e15.
doi: 10.1016/j.jtcvs.2021.09.010. Epub 2021 Sep 14.

Prediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores

Chin Siang Ong  1 , Erik Reinertsen  2 , Haoqi Sun  3 , Philicia Moonsamy  1 , Navyatha Mohan  1 , Masaki Funamoto  1 , Tsuyoshi Kaneko  4 , Prem S Shekar  4 , Stefano Schena  5 , Jennifer S Lawton  5 , David A D'Alessandro  1 , M Brandon Westover  6 , Aaron D Aguirre  7 , Thoralf M Sundt  1
Affiliations

Prediction of operative mortality for patients undergoing cardiac surgical procedures without established risk scores

Chin Siang Ong et al. J Thorac Cardiovasc Surg. 2023 Apr.

Abstract

Objective: Current cardiac surgery risk models do not address a substantial fraction of procedures. We sought to create models to predict the risk of operative mortality for an expanded set of cases.

Methods: Four supervised machine learning models were trained using preoperative variables present in the Society of Thoracic Surgeons (STS) data set of the Massachusetts General Hospital to predict and classify operative mortality in procedures without STS risk scores. A total of 424 (5.5%) mortality events occurred out of 7745 cases. Models included logistic regression with elastic net regularization (LogReg), support vector machine, random forest (RF), and extreme gradient boosted trees (XGBoost). Model discrimination was assessed via area under the receiver operating characteristic curve (AUC), and calibration was assessed via calibration slope and expected-to-observed event ratio. External validation was performed using STS data sets from Brigham and Women's Hospital (BWH) and the Johns Hopkins Hospital (JHH).

Results: Models performed comparably with the highest mean AUC of 0.83 (RF) and expected-to-observed event ratio of 1.00. On external validation, the AUC was 0.81 in BWH (RF) and 0.79 in JHH (LogReg/RF). Models trained and applied on the same institution's data achieved AUCs of 0.81 (BWH: LogReg/RF/XGBoost) and 0.82 (JHH: LogReg/RF/XGBoost).

Conclusions: Machine learning models trained on preoperative patient data can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. Such procedures comprise 23% of all cardiac surgical procedures nationwide. This work also highlights the value of using local institutional data to train new prediction models that account for institution-specific practices.

Keywords: cardiac surgery; machine learning; operative mortality; risk prediction.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Statement

The authors reported no conflicts of interest.

The Journal policy requires editors and reviewers to disclose conflicts of interest and to decline handling or reviewing manuscripts for which they may have a conflict of interest. The editors and reviewers of this article have no conflicts of interest.

Figures

FIGURE 1.
FIGURE 1.
Receiver operating characteristic (A) and calibration curves (B) of mortality risk models for cardiac surgical procedures without Society of Thoracic Surgeons risk scores at the Massachusetts General Hospital using preoperative variables. AUC, Area under the receiver operating characteristic curve; CI, confidence interval.
FIGURE 2.
FIGURE 2.
Machine learning can accurately predict mortality for procedures without STS scores.
FIGURE 3.
FIGURE 3.
Parsimonious model trained on subset of preoperative variables from the Massachusetts General Hospital (MGH; A; top variables, receiver operating characteristic and calibration curves shown). MGH-trained model applied to the Brigham and Women’s Hospital (BWH) cohort (B) and the Johns Hopkins Hospital (JHH) cohort (C); receiver operating characteristic and calibration curves shown. AUC, Area under the receiver operating characteristic curve; CI, confidence interval.
FIGURE 4.
FIGURE 4.
Machine learning (ML) models can predict operative mortality at a high level of accuracy for cardiac surgical procedures without established risk scores. MGH, Massachusetts General Hospital; BWH, Brigham and Women’s Hospital; AUC, area under the receiver operating characteristic curve; XGBoost, extreme gradient boosted trees.
See this image and copyright information in PMC

Comment in

References

    1. Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. Comparison of 19 preoperative risk stratification models in open-heart surgery. Eur Heart J. 2006;27:867–74. - PubMed
    1. O’Brien SM, Feng L, He X, Xian Y, Jacobs IP, Badhwar V, et al. The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 2-statistical methods and results. Ann Thorac Surg. 2018;105:1419–28. - PubMed
    1. Kilic A, Goyal A, Miller JK, Gjekmarkaj E, Tam WL, Gleason TG, et al. Predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery. Ann Thorac Surg. 2020;109:1811–9. - PubMed
    1. D’Agostino RS, Jacobs JP, Badhwar V, Fernandez FG, Paone G, Wormuth DW, et al. The Society of Thoracic Surgeons adult cardiac surgery database: 2018 update on outcomes and quality. Ann Thorac Surg. 2018;105:15–23. - PubMed
    1. Shahian DM, Jacobs JP, Badhwar V, Kurlansky PA, Furnary AP, Cleveland JC Jr, et al. The Society of Thoracic Surgeons 2018 adult cardiac surgery risk models: part 1-background, design considerations, and model development. Ann Thorac Surg. 2018;105:1411–8. - PubMed

Publication types