Integrating SHAP analysis with machine learning to predict postpartum hemorrhage in vaginal births

Abstract

Objective: This study aimed to develop a machine learning (ML) model integrated with SHapley Additive exPlanations (SHAP) analysis to predict postpartum hemorrhage (PPH) following vaginal deliveries, offering a potential tool for personalized risk assessment and prevention in clinical settings.

Methods: We conducted a retrospective multicenter cohort study in Northeast China, including women who had vaginal deliveries at three tertiary hospitals from September 2018 to December 2023. Data were extracted from electronic medical records. The dataset was split into a training set (70%) and an internal validation set (30%) to prevent overfitting. External validation was performed on a separate dataset. Several evaluation metrics, including the area under the receiver operating characteristic curve (AUC), were used to compare prediction performance. Features were ranked using SHAP, and the final model was explained.

Results: The XGBoost model demonstrated superior predictive accuracy for PPH, with an AUC of 0.997 in the training set. SHAP value-based feature selection identified 15 key features contributing to the model's predictive power. SHAP dependence and summary plots provided intuitive insights into each feature's contribution, enabling the identification of anomalies. The final model maintained high predictive power, with an AUC of 0.894 in internal validation and 0.880 in external validation.

Conclusion: This study successfully developed an interpretable ML model that predicts PPH with high accuracy. Future studies with larger and more diverse datasets are necessary to further validate and refine the model, particularly to assess its generalizability across different populations and healthcare settings.

Keywords: Interpretable model; Postpartum hemorrhage; SHAP; Vaginal births; XGBoost.

Fig. 1

Patient Selection Criteria Flowchart

Fig. 2

Comparison of ROC results of different machine learning models. XGBoost: eXtreme Gradient Boosting; LGBM: Light Gradient Boosting Machine; GBDT: Gradient Boosting Decision Tree; GBM: Gradient Boosting Machine; Ada: Adaptive Boosting; BNB: Bernoulli Naive Bayes

Fig. 3

Global model explanation of initial XGBoost model SHAP value for all risk factors. (A) SHAP summary bar plot. (B) SHAP summary dot plot

Fig. 4

Performance of XGBoost models to predict PPH. (A) AUC of the XGBoost model with varied numbers of features. (B) F1 score of the XGBoost model with varied numbers of features. (C) Pearson correlation plot of 15 features

Fig. 5

Global model explanation of final XGBoost model SHAP value for 15 risk factors. (A) SHAP summary bar plot. (B) SHAP summary dot plot

Fig. 6

SHAP dependence plot. Each dependence plot shows how a single feature affects the output of the prediction model, and each dot represents a single patient. The SHAP values for specific features exceeding zero push the decision towards the “PPH” class. LPL: light physical labor; MPL: moderate physical labor; HPL: heavy physical labor; PROM: premature rupture of membranes; BMI: body mass index

Fig. 7

Local model explanation by the SHAP method. (A) Waterfall plot of risks contributed by each feature for individual patient at low; (B) Waterfall plot of risks contributed by each feature for individual patient at high; (C) Force plot of risks contributed by each feature for individual patient at low; (D) Force plot of risks contributed by each feature for individual patient at high; (E) Evolution of risks contributed by each feature for individual patient at low; (F) Evolution of risks contributed by each feature for individual patient at high

Fig. 8

Model evaluation. (A) ROC of train cohort; (B) ROC of internal validation cohort; (C) ROC of external validation cohort; (D) calibration curve of train cohort; (E) DCA curve of train cohort; (F) calibration curve of internal validation cohort; (G) DCA curve of internal validation cohort; (H) calibration curve of external validation cohort; (I) DCA curve of external validation cohort. ROC: receiver operating characteristic curve; AUC: area under curve; DCA: decision curve analysis