Using a Predictive Risk Model to Prioritize Families for Prevention Services: The Hello Baby Program in Allegheny County, PA

Abstract

Population-based efforts to prevent child abuse and neglect are challenging because annual incidence rates are relatively low. Even among families that meet eligibility and risk criteria for intensive home-visiting programs, the baseline rate of maltreatment tends to be low because we use simple criteria. This creates both service (i.e., cost) and evaluation (i.e., power) challenges because a large number of families need to receive the preventive intervention to produce detectable changes in subsequent maltreatment. The increase in the availability of administrative data has made it possible to use predictive risk models (PRMs) to risk-stratify whole birth cohorts and identify children at the highest risk of maltreatment and other early childhood adversities. The current paper describes the development and validation of a PRM implemented in Allegheny County, Pennsylvania, to stratify families and newborn infants into three levels of prioritized services based on the predicted risk of child removal due to maltreatment by age 3. Using a research dataset of anonymized records for children born in Allegheny County between 2012 and 2015, predictive features were coded using data available in the county's administrative data systems. This spine was linked to child removal outcomes between 2012 and 2018, so we had a 3-year follow-up for each child. A PRM was trained to predict removals in the first 3 years of life using the least absolute shrinkage and selection operator. Predictive accuracy was measured for the highest 5% of risk scores in a holdout dataset. The model was validated using nontraining outcomes such as maternal mortality, infant mortality, and maltreatment-related fatalities and near-fatalities. The model achieved an area under the receiver operating characteristic curve of .93 (95% CI [0.92, 0.95]), recall of 19.93%, and precision of 54.10%. Children identified for the top tier of services had a relative risk ratio of maltreatment-related fatality or near-fatality of 5.54 (95% CI [3.41, 9.00]). Using alternative eligibility approaches (e.g., poverty, teen maternal age) proved far inferior to using PRM in targeting services for children at high baseline risk of maltreatment.

Keywords: Child maltreatment; Home visiting; Predictive risk model; Prevention.

Fig. 1

Prevalence of validation outcomes against the Hello Baby Predictive Risk Model Score (measured in 20 ventile score bins)

Fig. 2

Relative risk of adverse outcomes for counterfactual approaches to determine eligibility