Can Social Policies Improve Health? A Systematic Review and Meta-Analysis of 38 Randomized Trials

Abstract

Policy Points Social policies might not only improve economic well-being, but also health. Health policy experts have therefore advocated for investments in social policies both to improve population health and potentially reduce health system costs. Since the 1960s, a large number of social policies have been experimentally evaluated in the United States. Some of these experiments include health outcomes, providing a unique opportunity to inform evidence-based policymaking. Our comprehensive review and meta-analysis of these experiments find suggestive evidence of health benefits associated with investments in early life, income support, and health insurance interventions. However, most studies were underpowered to detect health outcomes.

Context: Insurers and health care providers are investing heavily in nonmedical social interventions in an effort to improve health and potentially reduce health care costs.

Methods: We performed a systematic review and meta-analysis of all known randomized social experiments in the United States that included health outcomes. We reviewed 5,880 papers, reports, and data sources, ultimately including 61 publications from 38 randomized social experiments. After synthesizing the main findings narratively, we conducted risk of bias analyses, power analyses, and random-effects meta-analyses where possible. Finally, we used multivariate regressions to determine which study characteristics were associated with statistically significant improvements in health outcomes.

Findings: The risk of bias was low in 17 studies, moderate in 11, and high in 33. Of the 451 parameter estimates reported, 77% were underpowered to detect health outcomes. Among adequately powered parameters, 49% demonstrated a significant health improvement, 44% had no effect on health, and 7% were associated with significant worsening of health. In meta-analyses, early life and education interventions were associated with a reduction in smoking (odds ratio [OR] = 0.92, 95% confidence interval [CI] 0.86-0.99). Income maintenance and health insurance interventions were associated with significant improvements in self-rated health (OR = 1.20, 95% CI 1.06-1.36, and OR = 1.38, 95% CI 1.10-1.73, respectively), whereas some welfare-to-work interventions had a negative impact on self-rated health (OR = 0.77, 95% CI 0.66-0.90). Housing and neighborhood trials had no effect on the outcomes included in the meta-analyses. A positive effect of the trial on its primary socioeconomic outcome was associated with higher odds of reporting health improvements. We found evidence of publication bias for studies with null findings.

Conclusions: Early life, income, and health insurance interventions have the potential to improve health. However, many of the included studies were underpowered to detect health effects and were at high or moderate risk of bias. Future social policy experiments should be better designed to measure the association between interventions and health outcomes.

Keywords: policy analysis; population health; randomized controlled trials; social determinants of health; social experiments.

Figure 1

PRISMA Flowchart of Sample Selection ^aThese searches included manual searches of the MDRC, Abt Associates, Mathematica, and RTI websites and the Digest of Social Experiments published by the Urban Institute in 2004. ^bStudies were excluded at this stage of the review for the following reasons: study was not a relevant intervention; no relevant health effects were quantified; study was not based in the United States; no abstract was electronically available (eg, conference proceedings or commentaries).

Figure 2

Trial Duration and Timing of Health Measurement (N = 38 Interventions)^{a a}For each trial, the shaded area indicates the trial duration in years, and the diamond (♦) indicates the latest health outcome measurement available.

Figure 3

Overview of Risk of Bias by Domain (N = 61 Studies)

Figure 4

Health Outcomes Reported (N = 61 studies)^{a a}Most studies included more than one health outcome. The contribution of each study to this graph is detailed in Online Appendix 4. ^b“Other” classification includes functional far vision, hay fever, pain, dental health, and vitality. ^cThe Framingham score is an algorithm used to estimate the 10‐year cardiovascular risk of an individual.

Figure 5

Overview of the Effects of Social Experiments on Health Outcomes by Policy Domain^{a a}No effect indicates a confidence interval that crosses 0 or P ≥.05. In Panel A, 451 estimates come from the 61 studies that compose our sample: 153 estimates in the early life and education domain, 45 in the income supplementation and maintenance domain, 52 in the employment and welfare‐to‐work domain, 110 in the housing and neighborhood domain, and 91 in health insurance domain. In Panel B, the 86 estimates are from those studies that are adequately powered (power ≥ 80%) to detect a health effect: 22 estimates in the early life and education domain, 3 in the income supplementation and maintenance domain, 13 in the employment and welfare‐to‐work domain, 7 in the housing and neighborhood domain, and 41 in the health insurance domain.

Figure 6

Meta‐analyses of the Effects of Social Experiments on Health^a Abbreviations: AB, Accelerated Benefits; CI, confidence interval; HCD, Human Capital Development group (includes skill training and education); LFA, Labor Force Attachment group (focused primarily on job search); OR, odds ratio; SRH, self‐rated health. ^aWeights are from random effects analyses. The diamond shape corresponds to the pooled odds ratio.