Use of Machine Learning to Develop a Risk-Stratification Tool for Emergency Department Patients With Acute Heart Failure

Abstract

Study objective: We use variables from a recently derived acute heart failure risk-stratification rule (STRATIFY) as a basis to develop and optimize risk prediction using additional patient clinical data from electronic health records and machine-learning models.

Methods: Using a retrospective cohort design, we identified all emergency department (ED) visits for acute heart failure between January 1, 2017, and December 31, 2018, among adult health plan members of a large system with 21 EDs. The primary outcome was any 30-day serious adverse event, including death, cardiopulmonary resuscitation, balloon-pump insertion, intubation, new dialysis, myocardial infarction, or coronary revascularization. Starting with the 13 variables from the STRATIFY rule (base model), we tested whether predictive accuracy in a different population could be enhanced with additional electronic health record-based variables or machine-learning approaches (compared with logistic regression). We calculated our derived model area under the curve (AUC), calculated test characteristics, and assessed admission rates across risk categories.

Results: Among 26,189 total ED encounters, mean patient age was 74 years, 51.7% were women, and 60.7% were white. The overall 30-day serious adverse event rate was 18.8%. The base model had an AUC of 0.76 (95% confidence interval 0.74 to 0.77). Incorporating additional variables led to improved accuracy with logistic regression (AUC 0.80; 95% confidence interval 0.79 to 0.82) and machine learning (AUC 0.85; 95% confidence interval 0.83 to 0.86). We found that 11.1%, 25.7%, and 48.9% of the study population had predicted serious adverse event risk of less than or equal to 3%, less than or equal to 5%, and less than or equal to 10%, respectively, and 28% of those with less than or equal to 3% risk were admitted.

Conclusion: Use of a machine-learning model with additional variables improved 30-day risk prediction compared with conventional approaches.