Explainable Preoperative Automated Machine Learning Prediction Model for Cardiac Surgery-Associated Acute Kidney Injury

Abstract

Background: We aimed to develop and validate an automated machine learning (autoML) prediction model for cardiac surgery-associated acute kidney injury (CSA-AKI).

Methods: Using 69 preoperative variables, we developed several models to predict post-operative AKI in adult patients undergoing cardiac surgery. Models included autoML and non-autoML types, including decision tree (DT), random forest (RF), extreme gradient boosting (XGBoost), and artificial neural network (ANN), as well as a logistic regression prediction model. We then compared model performance using area under the receiver operating characteristic curve (AUROC) and assessed model calibration using Brier score on the independent testing dataset.

Results: The incidence of CSA-AKI was 36%. Stacked ensemble autoML had the highest predictive performance among autoML models, and was chosen for comparison with other non-autoML and multivariable logistic regression models. The autoML had the highest AUROC (0.79), followed by RF (0.78), XGBoost (0.77), multivariable logistic regression (0.77), ANN (0.75), and DT (0.64). The autoML had comparable AUROC with RF and outperformed the other models. The autoML was well-calibrated. The Brier score for autoML, RF, DT, XGBoost, ANN, and multivariable logistic regression was 0.18, 0.18, 0.21, 0.19, 0.19, and 0.18, respectively. We applied SHAP and LIME algorithms to our autoML prediction model to extract an explanation of the variables that drive patient-specific predictions of CSA-AKI.

Conclusion: We were able to present a preoperative autoML prediction model for CSA-AKI that provided high predictive performance that was comparable to RF and superior to other ML and multivariable logistic regression models. The novel approaches of the proposed explainable preoperative autoML prediction model for CSA-AKI may guide clinicians in advancing individualized medicine plans for patients under cardiac surgery.

Keywords: AKI; acute kidney injury; artificial intelligence; cardiac surgery; cardiac surgery-associated acute kidney injury; individualized medicine; machine learning; personalized medicine; preoperative.

Figure 1

Comparison of AUROC among autoML model, different ML models, and logistic regression model. AUROC, area under the receiver operating characteristic curve; ML, machine learning.

Figure 2

Calibration plot autoML. Brier: Brier score; C (ROC), AUC for discrimination; D, discrimination index; Dxy, Somer’s rank correlation; Emax/E90/Eavg: Maximum/90th quantile, average absolute difference in predicted and smoothed calibrated probabilities; Q, quality index; R2: Nagelkerke-Cox-Snell-Maddala-Magee R-squared index; S:z/S:p the z and two-sided p-value of the Spiegelhalter test for calibration accuracy; U, unreliability index.

Figure 3

SHAP summary plot of the top 20 features of the GBM autoML (model ID: GBM_1_AutoML_1_20211031_170047), which is one of the key models in the component of our top autoML model. The higher the SHAP value of a feature, the higher the probability of CSA-AKI. Abbreviations: BMI, body mass index; BUN, blood urea nitrogen; eGFR, estimated glomerular filtration rate; pO2, partial pressure of oxygen; RVSP, right ventricular systolic pressure; SBP, systolic blood pressure.

Figure 4

Local interpretable model explainer (LIME) of top autoML (model ID: StackedEnsemble_AllModels_3_AutoML_1_20211031_170047) for six individual cases (case# 1 to 6) from the testing dataset. Label “1” means prediction of CSA-AKI and label “0” means prediction of no CSA-AKI. Probability shows the probability of the observation belong to the label “1” or “0”. The five most important features that best explain the linear model in that observation’s local region are demonstrated along with whether the features influence an increase in the probability (blue bar/supports or a decrease in the probability (red bar/contradicts). The x-axis demonstrated how much each feature added or subtracted to the final probability value for the patient. Abbreviations: BUN, blood urea nitrogen; eGFR, estimated glomerular filtration rate.