Construction and evaluation of machine learning-based prediction model for live birth following fresh embryo transfer in IVF/ICSI patients with polycystic ovary syndrome

Abstract

Objective: To investigate the determinants affecting live birth outcomes in fresh embryo transfer among polycystic ovary syndrome (PCOS) patients using various machine learning (ML) algorithms and to construct predictive models, offering novel insights for enhancing live birth rates in this specific group.

Methods: A sum of 1,062 fresh embryo transfer cycles involving PCOS patients were analyzed, with 466 resulting in live births. The dataset was split randomly into training and testing subsets at a 7:3 ratio. Least absolute shrinkage and selection operator and recursive feature elimination methods were utilized for feature selection within the training data. A grid search strategy identified the optimal parameters for seven ML models: decision tree (DT), K-nearest neighbors (KNN), light gradient boosting machine (LightGBM), naive Bayes model(NBM), random forest (RF), support vector machine (SVM) and extreme gradient boosting (XGBoost). The evaluation of model effectiveness incorporated diverse metrics, encompassing area under the curve (AUC), accuracy, positive predictive value, negative predictive value, F1 score, and Brier score. Calibration curves and decision curve analysis were employed to ascertain the optimal model. Furthermore, Shapley additive explanations were applied to elucidate the importance of predictor variables in the top-performing model.

Results: The AUC values of DT, KNN, LightGBM, NBM, RF, SVM and XGBoost models in the training set were 0.813, 1.000, 0.724, 0.791, 1.000, 0.819 and 0.853, respectively. Corresponding values in the testing set were 0.773, 0.719, 0.705, 0.764, 0.794, 0.806 and 0.822. XGBoost emerged as the most effective ML model. SHAP analysis revealed that variables encompassing embryo transfer count, embryo type, maternal age, infertility duration, body mass index, serum testosterone (T) levels, and progesterone (P) levels on the day of human chorionic gonadotropin administration were pivotal predictors of live birth outcomes in individuals with PCOS receiving fresh embryo transfer.

Conclusion: This study developed a live birth prediction model tailored for PCOS fresh embryo transfer cycles, leveraging ML algorithms to compare the efficacy of multiple models. The XGBoost model demonstrated superior predictive capacity, enabling prompt and precise identification of critical risk factors influencing live birth outcomes in PCOS patients. These findings offer actionable insights for clinical intervention, guiding strategies to improve pregnancy outcomes in this population.

Clinical trial number: Not applicable.

Keywords: Fresh embryo transfer; Live birth; Machine learning; Polycystic ovary syndrome.

Fig. 1

Flow chart of the study

Fig. 2

Features selected by LASSO and RFE. (A) The Lasso regression coefficient profiles of all baseline characteristics. (B) The optimal lambda selection in the Lasso regression with 10-fold cross-validation. Misclassification errors of different variables against log(lambda) are revealed. The two vertical dashed lines represent the optimal value under the minimum criterion and 1-SE criterion, respectively. The “lambda”is the tuning parameter. (C) A total of 9 predictors with non-zero coefficients are identified. (D) Features selected by RFE, When the number of features is 10, the RMSE is the lowest. (E) The top ten significant predictors identified by RFE. (F) The Venn diagram of features selected by LASSO and RFE. The intersection results of two methods yield 7 predictors. LASSO, Least Absolute Shrinkage and Selection Operator; RFE, Recursive Feature Elimination; RMSE, Root Mean Square Error

Fig. 3

Comparison of receiver operator characteristic curves (ROCs) for the machine learning models. (A) The ROCs of training models. (B) The ROCs of validation models. AUC, area under the ROC; DT, decision tree; KNN, k-nearest neighbors; LGBM: light gradient boosting machine; NBM, naïve bayes model; RF, random forest; XGBoost, eXtreme gradient boosting

Fig. 4

Discriminative power and accuracy of XGBoost model. A. The calibration curves of the validation group in XGBoost model. B The clinical decision curves of the validation group in XGBoost model

Fig. 5

SHAP plots. (A) SHAP summary plot shows feature importance for each predictor of the XGBoost model in descending order. The upper predictors are more important to the model’s predictive outcome. A dot is created for each feature attribution value for the XGBoost model of each patient. The further away a dot is from the baseline SHAP value of zero, the stronger it effects the model output. Dots are colored according to the values of features. Yellow represents higher feature values and red represents lower feature values. (B) Bar chart of the mean absolute SHAP value for each predictor of the XGBoost model in descending order. C and D. The force plots provide personalized feature attributions using two examples