Performance of an AI prediction tool for new-onset atrial fibrillation after coronary artery bypass grafting

Abstract

Background: There is lack of tools to predict new-onset postoperative atrial fibrillation (NOAF) after coronary artery bypass grafting (CABG). We aimed to develop and validate a novel AI-based bedside tool that accurately predicts predict NOAF after CABG.

Methods: Data from 2994 patients who underwent CABG between March 2015 and July 2024 at two tertiary hospitals in China were retrospectively analyzed. 2486 patients from one hospital formed the derivation cohort, split 7:3 into training and test sets, while the 508 patients from a separate hospital formed the external validation cohort. A stacking model integrating 11 base learners was developed and evaluated using Accuracy, Precision, Recall, F1 score, and Area Under Curve (AUC). SHapley Additive exPlanations (SHAP) values were calculated and plotted to interpret the contributions of individual characteristics to the model's predictions.

Findings: Seventy-seven predictive characteristics were analyzed. The stacking model achieved superior performance with AUCs 0·931 and F1 scores 0·797 in the independent external validation, outperforming CHA2DS2-VASc, HATCH, and POAF scores (AUC 0·931 vs. 0·713, 0·708, and 0·667; p < 0·05). SHAP value indicate that the importance of predictive features for NOAF, in descending order, include: Brain natriuretic peptide, Left ventricular end-diastolic diameter, Ejection fraction, BMI, β-receptor blockers, Duration of surgery, Age, Neutrophil percentage-to-albumin ratio, Myocardial infarction, Left atrial diameter, Hypertension, and smoking status. Subsequently, we constructed an easy-to-use bedside clinical tool for NOAF risk assessment leveraging these characteristics.

Interpretation: The AI-based tool offers superior prediction of NOAF, outperforming three existing predictive tools. Future studies should further explore how various patient characteristics influence the timing of NOAF onset, whether early or late.

Funding: This work was funded by Lingnan Nightingale Nursing Research Institute of Guangdong Province, and Guangdong Nursing Society (GDHLYJYZ202401).

Keywords: Artificial intelligence; Coronary artery bypass grafting; Machine learning; Postoperative atrial fibrillation; Prediction model; Web tool.

Fig. 1

Note: ABC, Adaptive Boosting Classifier; ALB, albumin; AUC, area under curve; BNP, brain natriuretic peptide; BRB, β-receptor blockers; CABG, coronary artery bypass grafting; CBC, CatBoost Classifier; DCA, decision curve analysis; DTC, Decision Tree Classifier; H, height; KNN, K-Nearest Neighbors; LAD, left atrial diameter; LGB, Light Gradient Boosting Machine; LR, Logistic regression; LVEF, left ventricular ejection fraction; LVEDD, left ventricular end diastolic diameter; MI, myocardial infarction; MLP, Multilayer Perceptron; NB, Naive Bayes; NE, neutrophil; PCC, probability calibration curve; RF, Random Forest; ROC, receiver operating characteristic curve; SVC, Support Vector Classifier; W, weight; XGB, eXtreme Gradient Boosting. Part A: This is a prognostic study, the first phase of which involved developing a predictive model for NOAF in patients undergoing CABG using 11 base learners stacked via a stacking technique, with internal and external validation. The corresponding results are presented in Fig. 3. Part B: The second phase focused on creating a user-friendly bedside tool. The tool features a user-friendly interface. A prompt, explaining the abbreviations and units for all data inputs (e.g., BNP: brain natriuretic peptide (pg/ml), etc.), will appear automatically every time the user opens the tool and cannot be closed for 5 s. Medical professionals can simply input the relevant parameters and click “Calculate New-onset POAF for CABG Patient” to determine the probability of NOAF occurrence. If the probability of NOAF exceeds 95% or falls below 5%, our tool provides a prompt saying, “The prediction probability is too [HIGH/LOW], please check the numerical inputs carefully, especially the units.”.

Fig. 2

Note: A/G, ALB/GLB ratio; ALB, albumin; ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate aminotransferase; ADS, antibiotics during surgery; BMI, body mass index; BNP, brain natriuretic peptide; BRB, β-receptor blockers; BS, blood sugar; CCB, calcium channel blocker; CI, confidence interval; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; D-Bil, direct bilirubin; DBP, diastolic blood pressure; GGT, gamma-glutamyl transferase; GLB, globulin; Hb, hemoglobin; HDL-C, high-density lipoprotein cholesterol; I < O, input is less than output; I-Bil, indirect bilirubin; LAD, left atrial diameter; LASSO, least absolute shrinkage and selection operator; LDL-C, low-density lipoprotein cholesterol; LVEF, left ventricular ejection fraction; LVEDD, left ventricular end diastolic diameter; MI, myocardial infarction; MPV, mean platelet volume; NPAR, neutrophil percentage-to-albumin ratio; NYHA, New York Heart Association; OR, odds ratio; PLT, platelet; PVD, peripheral vascular disease; RBC, red blood cell; RDW, red cell volume distribution width; SBP, systolic blood pressure; SMOTE, synthetic minority oversampling technique; T-Bil, total bilirubin; TC, total cholesterol; TG, triglyceride; TP, total protein; WBC, white blood cell. Part A: Participant flow chart. Finally, a total of 77 predictive characteristics were collected, categorized into eight groups. Part B: Twenty-one features were included in the LASSO regression. The dashed line in the figure indicates the optimal alpha value, with 12 features to the left having non-zero coefficients, proceeding to the next selection step. Part C: Twelve features were subjected to further selection using logistic regression, with results available in Supplementary File part 9. The statistically significant twelve features underwent SMOTE processing to address the imbalance in NOAF positive samples, resulting in the formation of the final dataset E.

Fig. 3

Note: AUC, area under curve; BNP, brain natriuretic peptide; BRB, β-receptor blockers; CBC, CatBoost Classifier; DCA, decision curve analysis; DTC, Decision Tree Classifier; FPR, false positive rate; KNN, K-Nearest Neighbors; LAD, left atrial diameter; LGB, Light Gradient Boosting Machine; LR, Logistic regression; LVEF, left ventricular ejection fraction; LVEDD, left ventricular end diastolic diameter; MI, myocardial infarction; MLP, Multilayer Perceptron; NB, Naive Bayes; PCC, probability calibration curve; RF, Random Forest; ROC, receiver operating characteristic curve; SVC, Support Vector Classifier; TPR, true positive rate; XGB, eXtreme Gradient Boosting; NPAR, neutrophil percentage-to-albumin ratio. Part A: The figure illustrates the maintenance rate of sinus rhythm after coronary artery bypass grafting, indicating that new-onset atrial fibrillation (NOAF) is prevalent in the first 1–5 days postoperatively. Part B: This is a Shapley Additive Explanations (SHAP) diagram. The twelve features presented effectively predict NOAF. The features' importance for predicting NOAF increases as they are ranked higher in the diagram, with preoperative BNP levels being the most significant and smoking history the least significant. Part C: The left side presents the ROC curves of the internal validation set, with each curve representing a different model. The closer a curve is to the top-left corner of the box, the better the model's performance. It is evident that our Stacking model demonstrates the best performance. Furthermore, all our models outperform the existing three tools. The right side displays the clinical decision curves for the internal validation set, where each curve also signifies a model. Again, curves closer to the top-right corner indicate superior model performance, confirming that our Stacking model is the best. Part D: Similar to Part C, this figure illustrates the performance of all models on the external validation set, indicating that our Stacking model performs the best. Part E: This figure illustrates the probability calibration curve, with the blue line representing the performance of the Stacking model; the closer it is to the dashed line, the better the model's performance. The p-value from the Hosmer–Lemeshow test is 0·217, which exceeds 0·05, indicating no significant evidence of systematic bias between the model's predicted probabilities and the observed actual outcomes, thus reflecting good model performance.