Evaluating the performance of an AI-powered VBAC prediction system within a decision-aid birth choice platform for shared decision-making

Abstract

Background: Vaginal birth after cesarean (VBAC) is generally regarded as a safe and viable birthing option for most women with prior cesarean delivery. Nonetheless, concerns about heightened risks of adverse maternal and perinatal outcomes have often dissuaded women from considering VBAC. This study aimed to assess the performance of an artificial intelligence (AI)-powered VBAC prediction system integrated into a decision-aid birth choice platform for shared decision-making (SDM).

Materials and methods: Employing a retrospective design, we collected medical records from a regional hospital in northern Taiwan from January 2019 to May 2023. To explore a suitable model for tabular data, we compared two prevailing modeling approaches: tree-based models and logistic regression models. We subjected the tree-based algorithm, CatBoost, to binary classification.

Results: Forty pregnant women with 347 records were included. The CatBoost model demonstrated a robust performance, boasting an accuracy rate of 0.91 (95% confidence interval (CI): 0.86-0.94) and an area under the curve of 0.89 (95% CI: 0.86-0.93), surpassing both regression models and other boosting techniques. CatBoost captured the data characteristics on the significant impact of gravidity and the positive influence of previous vaginal birth, reinforcing established clinical guidelines, as substantiated by the SHapley Additive exPlanations analysis.

Conclusion: Using AI techniques offers a more accurate assessment of VBAC risks, boosting women's confidence in selecting VBAC as a viable birthing option. The seamless integration of AI prediction systems with SDM platforms holds a promising potential for enhancing the effectiveness of clinical applications in the domain of women's healthcare.

Keywords: Vaginal birth after cesarean; artificial intelligence prediction; elective repeat cesarean delivery; pregnant women; shared decision making.

Figure 1.

Web-based decision-aid platform comprising four components: The Interactive Web Service (IWS) facilitated shares decision-making between doctors and pregnant women and provides decision aids. The platform uses Responsive Web Design (RWD) to ensure optimal performance on personal computers, tablets, and mobile devices. The RWD also supports the storage of pregnant women's medical history and physiological data, which are achieved through integration with the VBAC database system on a cloud server. The VBAC prediction system utilizes cloud-based AI technology to compute and employ a predictive model that determines the probability of a successful vaginal birth during pregnancy.

Figure 2.

AI prediction with medical record form and ML model.

Figure 3.

Recruitment flow chart for the AI model.

Figure 4.

Confusion matrix for every ML model, including logistic regression, random forest, XGBoost, LightGBM, decision tree, and CatBoost. 0: VBAC failure group. 1: VBAC success group.

Figure 5.

ROC curve for the performance evaluation at every ML model, including logistic regression, random forest, XGBoost, LightGBM, decision tree, and CatBoost.

Figure 6.

Feature importance based on SHAP for every ML model, including logistic regression, random forest, XGBoost, LightGBM, decision tree, and CatBoost.