Predicting and Preventing Gun Violence: An Experimental Evaluation of READI Chicago

Abstract

Gun violence is the most pressing public safety problem in American cities. We report results from a randomized controlled trial ($N=2,456$) of a community-researcher partnership called the Rapid Employment and Development Initiative (READI) Chicago. The program offered an 18-month job alongside cognitive behavioral therapy and other social support. Both algorithmic and human referral methods identified men with strikingly high scope for gun violence reduction: for every 100 people in the control group, there were 11 shooting and homicide victimizations during the 20-month outcome period. Fifty-five percent of the treatment group started programming, comparable to take-up rates in programs for people facing far lower mortality risk. After 20 months, there is no statistically significant change in an index combining three measures of serious violence, the study's primary outcome. Yet there are signs that this program model has promise. One of the three measures, shooting and homicide arrests, declines 65 percent ($p=0.13$ after multiple testing adjustment). Because shootings are so costly, READI generates estimated social savings between $182,000 and $916,000 per participant ($p=0.03$), implying a benefit-cost ratio between 4:1 and 18:1. Moreover, participants referred by outreach workers-a pre-specified subgroup-show enormous declines in both arrests and victimizations for shootings and homicides (79 and 43 percent, respectively) that remain statistically significant even after multiple testing adjustments. These declines are concentrated among outreach referrals with higher predicted risk, suggesting that human and algorithmic targeting may work better together.

Figure A.I

READI job stage progression Notes: The duration and advancement requirements for READI’s job stages were subject to change based on a participant’s needs and progress. Program staff exercised discretion in deciding which participants were ready to advance job stages. The diagram shows READI’s initial design; the details of implementation varied somewhat in practice as the model developed over time.

Figure A.II

READI wage growth by pathway Notes: READI’s starting wage was $11 at its launch in August 2017, increased to $12 in July 2018, and to $13 in July 2019. Average wage is calculated using only participants who report to work during a given week.

Figure A.III

READI job retention, overall and by pathway, including COVID period Notes: Figure shows two measures of job retention for men who started READI measured from payroll data. The solid line shows the proportion of participants who work at least once after the time shown on the x-axis conditional on observing them for that long. The boxes show the number of workers contributing to each point. The dotted line shows the average proportion of possible weeks worked among those still working at each point in time. At 18 months after first taking up, N = 38 algorithm referrals, N = 68 outreach referrals, and N = 17 re-entry referrals are still observed working.

Figure A.IV

Predicted and actual dosage for treatment participants Notes: Figure shows dosage predictions from an endogenous stratification exercise that uses a leave-one-out regression of hours participated on observable baseline covariates among the treatment group. X-axis is the average predicted hours participated in each decile bin. Y-axis is the average true hours participated in that bin. The solid line is the 45 degree line that would reflect good calibration across the distribution. Appendix Section A.6.5 discusses the use of these predictions to analyze treatment heterogeneity by predicted dosage for the entire sample.

Figure A.V

Cumulative first stage and ITT effects over time Notes: Figures show cumulative treatment effects up to the time shown on the x-axis, inclusive. Top left panel shows indicators for any participation; other panels show the three main components of the primary index. Regressions include baseline covariates and randomization strata fixed effects, and 95 percent confidence intervals are constructed using heteroskedasticity-robust standard errors.

Figure A.VI

Distribution of risk scores by pathway and estimated effects on index components by pathway and risk level Notes: Top left panel shows distributions of the risk score, the predicted probability at baseline of being a victim or an arrestee in a violent gun crime during the next 18 months, by pathway. Shaded areas denote quartiles of the risk score. The remaining panels show coefficient estimates and 95 percent confidence intervals (using heteroskedasticity-robust standard errors) on three-way interactions of pathway indicators, risk quartile indicators, and an indicator for being randomized to receive a READI offer, from regressions of the primary index components on baseline covariates, randomization strata fixed effects, and all two-way interactions of pathway indicators, risk quartile indicators, and an indicator for being randomized to receive a READI offer.

Figure I

Shooting victims per 100,000 residents (2016), by neighborhood Notes: Plot shows counts and rates of shooting and homicide victims in each of Chicago’s 77 neighborhoods in 2016. Dashed lines represent top 15 neighborhoods for each dimension..

Figure II

READI job retention, overall and by pathway Notes: Figure shows two measures of job retention for men who started READI measured from payroll data. The solid line shows the proportion of participants who work at least once after the time shown on the x-axis conditional on observing them for that long. The boxes show the number of workers contributing to each point. The dotted line shows the average proportion of possible weeks worked among those still working at each point in time. At 18 months after first taking up, N = 19/60 algorithm referrals and N = 38/94 outreach referrals are still observed working. Since COVID-19 changed both what “participation” looked like in practice as well as payroll policies, we report these measures of retention using only data through the start of the pandemic. Because there were too few re-entry participants with sufficient pre-COVID data to measure retention, they are omitted. For a description of the pandemic’s impact on READI, see Section 2.3 and Appendix A.5.3. For retention measures inclusive of the pandemic period, see Appendix Figure A.III..

Figure III

READI estimated effects, realized risk, and take-up by pathway and predicted risk Notes: The top panel shows coefficient estimates and 95 percent confidence intervals (using heteroskedasticity-robust standard errors) on three-way interactions of pathway indicators, risk score quartile indicators, and an indicator for being randomized to receive a READI offer, from a regression of the primary index on baseline covariates, randomization strata fixed effects, and all two-way interactions of pathway indicators, risk score quartile indicators, and an indicator for being randomized to receive a READI offer. Bottom left panel shows the realized rate of involvement in a violent gun crime as a victim or an arrestee during the 18 months after randomization by quartile of the risk score, which is the predicted probability at baseline of the same outcome, separately for algorithm and outreach referrals. Bottom right panel shows the take-up rate, defined as the share of the treatment group attending the first day of READI orientation, by quartile of the risk score, separately for algorithm and outreach referrals. Markers in both of the bottom panels are weighted to reflect the share of algorithm and outreach referrals in each risk score quartile. N = 231 for the missing risk score group (of whom 161 are outreach referrals). For ITT estimates of each individual index component by pathway and risk score quartile, see Appendix Figure A.VI.