The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains

Abstract

Over the past decade, choice architecture interventions or so-called nudges have received widespread attention from both researchers and policy makers. Built on insights from the behavioral sciences, this class of behavioral interventions focuses on the design of choice environments that facilitate personally and socially desirable decisions without restricting people in their freedom of choice. Drawing on more than 200 studies reporting over 450 effect sizes (n = 2,149,683), we present a comprehensive analysis of the effectiveness of choice architecture interventions across techniques, behavioral domains, and contextual study characteristics. Our results show that choice architecture interventions overall promote behavior change with a small to medium effect size of Cohen's d = 0.45 (95% CI [0.39, 0.52]). In addition, we find that the effectiveness of choice architecture interventions varies significantly as a function of technique and domain. Across behavioral domains, interventions that target the organization and structure of choice alternatives (decision structure) consistently outperform interventions that focus on the description of alternatives (decision information) or the reinforcement of behavioral intentions (decision assistance). Food choices are particularly responsive to choice architecture interventions, with effect sizes up to 2.5 times larger than those in other behavioral domains. Overall, choice architecture interventions affect behavior relatively independently of contextual study characteristics such as the geographical location or the target population of the intervention. Our analysis further reveals a moderate publication bias toward positive results in the literature. We end with a discussion of the implications of our findings for theory and behaviorally informed policy making.

Keywords: behavior change; behavioral insights; choice architecture; meta-analysis; nudge.

Fig. 1.

Number of citations of Thaler and Sunstein (1) between 2008 and 2020. Counts are based on citation search in Web of Science.

Fig. 2.

Forest plot of all effect sizes (k = 455) included in the meta-analysis with their corresponding 95% confidence intervals. Extracted Cohen’s d values ranged from –0.69 to 4.69. The proportion of true to total variance was estimated at I² = 99.67%. ***$P<\mathrm{0.001.}$

Fig. 3.

Funnel plot displaying each observation as a function of its effect size and SE. In the absence of publication bias, observations should scatter symmetrically around the pooled effect size indicated by the gray vertical line and within the boundaries of the 95% confidence intervals shaded in white. The asymmetric distribution shown here indicates a one-tailed publication bias in the literature that favors the reporting of successful implementations of choice architecture interventions in studies with small sample sizes.

Fig. 4.

Forest plot of effect sizes across categories of choice architecture intervention techniques (see Table 1 for more detailed description of techniques). The position of squares on the x axis indicates the effect size of each respective intervention technique. Bars indicate the 95% confidence intervals of effect sizes. The size of squares is inversely proportional to the SE of effect sizes. Diamond shapes indicate the average effect size and confidence intervals of intervention categories. The solid line represents an effect size of Cohen’s d = 0. The dotted line represents the overall effect size of choice architecture interventions, Cohen’s d = 0.45, 95% CI [0.39, 0.52]. Identical letter superscripts indicate statistically significant (P < 0.05) pairwise comparisons.

Fig. 5.

Forest plot of effect sizes across categories of choice architecture interventions and behavioral domains. The position of squares on the x axis indicates the effect size of each intervention category within a behavioral domain. Bars indicate the 95% confidence intervals of effect sizes. The size of squares is inversely proportional to the SE of effect sizes. Diamond shapes indicate the overall effect size and confidence intervals of choice architecture interventions within a behavioral domain. The solid line represents an effect size of Cohen’s d = 0. The dotted line represents the overall effect size of choice architecture interventions, Cohen’s d = 0.45, 95% CI [0.39, 0.52]. Identical letter superscripts indicate statistically significant (P < 0.05) pairwise comparisons.