Predictive Modeling to Identify Children With Complex Health Needs At Risk for Hospitalization

Abstract

Background: Identifying children at high risk with complex health needs (CCHN) who have intersecting medical and social needs is challenging. This study's objectives were to (1) develop and evaluate an electronic health record (EHR)-based clinical predictive model ("model") for identifying high-risk CCHN and (2) compare the model's performance as a clinical decision support (CDS) to other CDS tools available for identifying high-risk CCHN.

Methods: This retrospective cohort study included children aged 0 to 20 years with established care within a single health system. The model development/validation cohort included 33 months (January 1, 2016-September 30, 2018) and the testing cohort included 18 months (October 1, 2018-March 31, 2020) of EHR data. Machine learning methods generated a model that predicted probability (0%-100%) for hospitalization within 6 months. Model performance measures included sensitivity, positive predictive value, area under receiver-operator curve, and area under precision-recall curve. Three CDS rules for identifying high-risk CCHN were compared: (1) hospitalization probability ≥10% (model-predicted); (2) complex chronic disease classification (using Pediatric Medical Complexity Algorithm [PMCA]); and (3) previous high hospital utilization.

Results: Model development and testing cohorts included 116 799 and 27 087 patients, respectively. The model demonstrated area under receiver-operator curve = 0.79 and area under precision-recall curve = 0.13. PMCA had the highest sensitivity (52.4%) and classified the most children as high risk (17.3%). Positive predictive value of the model-based CDS rule (19%) was higher than CDS based on the PMCA (1.9%) and previous hospital utilization (15%).

Conclusions: A novel EHR-based predictive model was developed and validated as a population-level CDS tool for identifying CCHN at high risk for future hospitalization.

FIGURE 1

Screenshot from online dashboard built for clinicians to visualize data model predictions.

FIGURE 2

Data flow diagram. N refers to counts of individual patients in each sample.

FIGURE 3

(A) Area under receiver operator curve (AUROC) of hospitalization risk scores for primary care test clinic patients. (B) Area under precision recall curve (AUPRC) of hospitalization risk scores for primary care test clinic patients. AUROC calculated using data from 27 087 patients included in the primary care testing cohort as of 9/1/2019.

FIGURE 4

Overlap of high-risk patients identified by data model, PMCA, and previous high hospital utilization. Data represent a cross-section of 3856 children attributed to the primary care test clinic as of September 1, 2019, to which the 3 clinical decision support (CDS) rules under evaluation were applied.