Improving postpartum hemorrhage risk prediction using longitudinal electronic medical records

Abstract

Objective: Postpartum hemorrhage (PPH) remains a leading cause of preventable maternal mortality in the United States. We sought to develop a novel risk assessment tool and compare its accuracy to tools used in current practice.

Materials and methods: We used a PPH digital phenotype that we developed and validated previously to identify 6639 PPH deliveries from our delivery cohort (N = 70 948). Using a vast array of known and potential risk factors extracted from electronic medical records available prior to delivery, we trained a gradient boosting model in a subset of our cohort. In a held-out test sample, we compared performance of our model with 3 clinical risk-assessment tools and 1 previously published model.

Results: Our 24-feature model achieved an area under the receiver-operating characteristic curve (AUROC) of 0.71 (95% confidence interval [CI], 0.69-0.72), higher than all other tools (research-based AUROC, 0.67 [95% CI, 0.66-0.69]; clinical AUROCs, 0.55 [95% CI, 0.54-0.56] to 0.61 [95% CI, 0.59-0.62]). Five features were novel, including red blood cell indices and infection markers measured upon admission. Additionally, we identified inflection points for vital signs and labs where risk rose substantially. Most notably, patients with median intrapartum systolic blood pressure above 132 mm Hg had an 11% (95% CI, 8%-13%) median increase in relative risk for PPH.

Conclusions: We developed a novel approach for predicting PPH and identified clinical feature thresholds that can guide intrapartum monitoring for PPH risk. These results suggest that our model is an excellent candidate for prospective evaluation and could ultimately reduce PPH morbidity and mortality through early detection and prevention.

Keywords: clinical decision support; electronic medical records; phenotype; postpartum hemorrhage; risk assessment.

Figure 1.

Overview of study design and model development. AWHONN: Association of Women’s Health, Obstetric and Neonatal Nurses; CMQCC: California Maternal Quality Care Collaborative; CSLS: U.S. Consortium for Safe Labor Study; ICD: International Classification of Diseases; NYSBOH: New York Safety Bundle for Obstetric Hemorrhage; PPH: postpartum hemorrhage.

Figure 2.

SHAP summary plot. SHAP summary plot for top 24 clinical features for postpartum hemorrhage (PPH) prediction shows the SHAP values for the most important features from Gradient boosting model in the training data. Features in the summary plot (y-axis) are ordered by the mean absolute SHAP values (x-axis), which represents the importance of the feature in driving the PPH prediction. Values of the feature for each patient are colored by their relative value, with red color indicating high value and blue color indicating low value. Positive SHAP values indicate increased risks for PPH and negative values indicate protective effects to PPH. DBP: diastolic blood pressure; freq.: frequency; hosp.: hospital; SBP: systolic blood pressure.

Figure 3.

Dynamic changes of 3 vital signs consistently measured prior to delivery. Moving averages and standard deviations with 3-hour windows across the 12 hours prior to delivery were computed for cases and controls. PPH: postpartum hemorrhage.

Figure 4.

SHAP dependency plot. SHAP scores (relative risks, y-axis) for postpartum hemorrhage (PPH) prediction was plotted against feature values (x-axis) for patients in the training data. The plot shows how different values of the features can affect relative risks and ultimately impact classifier decision for 6 vital signs and lab measurements stratified by type of delivery. The shaded gray area reflects the reference ranges for the corresponding vital signs or lab measures. Data points are colored by the delivery method (Cesarean or vaginal). DBP: diastolic blood pressure; SBP: systolic blood pressure.

Figure 5.

Postpartum hemorrhage (PPH) prevalence among patients of different risk groups varies by risk assessment tools. Case prevalence within each risk category for each risk tool was calculated. Risk categories were assigned using deciles for U.S. Consortium for Safe Labor Study (CSLS) and Sema4 models (high risk = top 10%, medium risk = 60%-90%, low risk = <60%). AWHONN: Association of Women’s Health, Obstetric and Neonatal Nurses; CMQCC: California Maternal Quality Care Collaborative; IML: integrated machine learning; NYSBOH: New York Safety Bundle for Obstetric Hemorrhage.