Machine Learning and Statistical Models to Predict Postpartum Hemorrhage

Abstract

Objective: To predict a woman's risk of postpartum hemorrhage at labor admission using machine learning and statistical models.

Methods: Predictive models were constructed and compared using data from 10 of 12 sites in the U.S. Consortium for Safe Labor Study (2002-2008) that consistently reported estimated blood loss at delivery. The outcome was postpartum hemorrhage, defined as an estimated blood loss at least 1,000 mL. Fifty-five candidate risk factors routinely available on labor admission were considered. We used logistic regression with and without lasso regularization (lasso regression) as the two statistical models, and random forest and extreme gradient boosting as the two machine learning models to predict postpartum hemorrhage. Model performance was measured by C statistics (ie, concordance index), calibration, and decision curves. Models were constructed from the first phase (2002-2006) and externally validated (ie, temporally) in the second phase (2007-2008). Further validation was performed combining both temporal and site-specific validation.

Results: Of the 152,279 assessed births, 7,279 (4.8%, 95% CI 4.7-4.9) had postpartum hemorrhage. All models had good-to-excellent discrimination. The extreme gradient boosting model had the best discriminative ability to predict postpartum hemorrhage (C statistic: 0.93; 95% CI 0.92-0.93), followed by random forest (C statistic: 0.92; 95% CI 0.91-0.92). The lasso regression model (C statistic: 0.87; 95% CI 0.86-0.88) and logistic regression (C statistic: 0.87; 95% CI 0.86-0.87) had lower-but-good discriminative ability. The above results held with validation across both time and sites. Decision curve analysis demonstrated that, although all models provided superior net benefit when clinical decision thresholds were between 0% and 80% predicted risk, the extreme gradient boosting model provided the greatest net benefit.

Conclusion: Postpartum hemorrhage on labor admission can be predicted with excellent discriminative ability using machine learning and statistical models. Further clinical application is needed, which may assist health care providers to be prepared and triage at-risk women.

Figure 1.

Flow chart of women with postpartum hemorrhage (estimated blood loss [EBL] ≥1,000 mL).

Figure 2.

Calibration curves demonstrating the performance of predicting postpartum hemorrhage (PPH) for all four models: Logistic regression (A), logistic regression with lasso regularization (B), random forest (C), and extreme gradient boosting (D). The figure demonstrates the variation in each model’s performance. The red line indicates perfect agreement between the predicted probability of the model and the actual probability. The black line bounded by two dotted lines indicates the overall calibration with 95% CIs of each model. Each triangle represents a group of individuals risk. There are 10 triangles and each triangle represents a decile of risk. A. Calibration (intercept: –0.17 [–0.22 to –0.11]; slope: 0.96 [0.92 to 1.00]); discrimination (C-statistic: 0.87 [0.86 to 0.87]). B. Calibration (intercept: –0.20 [–0.26 to –0.15]; slope: 1.08 [1.04 to 1.12]); discrimination (C-statistic: 0.87 [0.86 to 0.88]). C. Calibration (intercept: –0.60 [–0.66 to –0.54]; slope: 1.28 [1.23 to 1.33]); discrimination (C-statistic: 0.92 [0.91 to 0.92]). D. Calibration (intercept: –0.13 [–0.19 to –0.06]; slope: 1.02 [0.98 to 1.06]); discrimination (C-statistic: 0.93 [0.92 to 0.94]).

Figure 3.

Decision curve analysis of predicting postpartum hemorrhage by all models. The x-axis indicates the threshold probability for postpartum hemorrhage outcome. The threshold probability is a level of certainty above which the patient or physician would choose to intervene. The probability threshold captures the relative value the patient or physician places on receiving an intervention for the outcome, if present, to the value of avoiding an intervention if the outcome is not present. The y-axis indicates the net benefit. The net benefit is calculated as true positive rate – (false positive rate x weighting factor). Weighting factor is calculated as threshold probability/1–threshold probability. For example, when threshold probability is 0.1, weighting factor is 0.1/1–0.1=0.1/0.9. The decision curves indicate the net benefit of each model as well as two clinical alternatives (classifying no women as having the outcome vs. classifying all women as having the outcome) over a specified range of threshold probabilities of outcome. Compared with the clinical alternatives, the net benefit for the Extreme Gradient Boosting model was greatest across the range of threshold probabilities.

Figure 4.

Importance of each predictor in the Extreme Gradient Boosted model to predict risk of postpartum hemorrhage. The variable importance is a measure scaled to have a maximum value of one. Cluster indicates features that are similar to one another in importance value. BMI, body mass index.