Mechanisms of implementing public health interventions: a pooled causal mediation analysis of randomised trials

Abstract

Background: The World Health Organization recommends that nations implement evidence-based nutritional guidelines and policies in settings such as schools and childcare services to improve public health nutrition. Understanding the causal mechanism by which implementation strategies exert their effects could enhance guideline implementation. The aim of this study was to assess the mechanisms by which implementation strategies improved schools and childcare services' adherence to nutrition guidelines.

Methods: We conducted a mechanism evaluation of an aggregated dataset generated from three randomised controlled trials conducted in schools and childcare services in New South Wales, Australia. Each trial examined the impact of implementation strategies that targeted Theoretical Domains Framework constructs including knowledge, skills, professional role and identity, environmental context and resources. We pooled aggregated organisation level data from each trial, including quantitative assessments of the Theoretical Domains Framework constructs, as well as measures of school or childcare nutrition guideline compliance, the primary implementation outcome. We used causal mediation analysis to estimate the average indirect and direct effects of the implementation strategies and assessed the robustness of our findings to varying levels of unmeasured and unknown confounding.

Results: We included 121 schools or childcare services in the pooled analysis: 79 allocated to receive guideline and policy implementation strategies and 42 to usual practice. Overall, the interventions improved compliance (odds ratio = 6.64; 95% CI [2.58 to 19.09]); however, the intervention effect was not mediated by any of the four targeted Theoretical Domains Framework constructs (average causal mediation effects through knowledge = - 0.00 [- 0.05 to 0.04], skills = 0.01 [- 0.02 to 0.07], professional role and identity = 0.00 [- 0.03 to 0.03] and environmental context and resources = 0.00 [- 0.02 to 0.06]). The intervention had no significant effect on the four targeted Theoretical Domains Framework constructs, and the constructs were not associated with school or childcare nutrition guideline compliance. Potentially, this lack of effect could be explained by imprecise measurement of the mediators. Alternatively, it is likely that that the interventions were operating via alternative mechanisms that were not captured by the four Theoretical Domains Framework constructs we explored.

Conclusions: Even though public health implementation strategies led to meaningful improvements in school or childcare nutrition guideline compliance, these effects were not mediated by key targeted constructs of the Theoretical Domains Framework. Future research should explore the mechanistic role of other Theoretical Domains Framework constructs and evaluate system-level mechanisms informed by an ecological framework.

Trial registration: All trials were prospectively registered with the Australian New Zealand Clinical Trials Registry ( ACTRN12613000543785 . Registered 15/05/2013; ACTRN12614001148662 . Registered 30/10/2014; ACTRN12615001032549 . Registered 1/10/2015).

Keywords: Implementation science; Mechanism; Mediation analysis; Public health; Theoretical domains framework.

Fig. 1

Directed acyclic graph of hypothesised mechanisms. Blue arrows = average causal mediation effect, black arrow = average direct effect, green arrow = intervention-mediator interaction, red arrows = confounding effects. Here we assume that the four mediators are independent of one another

Fig. 2

Count of organisations that improved or did not change practice, stratified by trial

Fig. 3

Effect decomposition plots for each mediator model. ACME average causal mediation effect, ADE average direct effect. Solid dots and lines represent point estimates and 95% confidence limits for the intervention group; the hollow dots and broken lines represent point estimates and confidence limits for the usual care group. The total effect is displayed as an average effect. All effects are reported unstandardized with their 95% confidence intervals

Fig. 4

Sensitivity plots. The average mediation effects are plotted as a function of the sensitivity parameter (magnitude of residual confounding). A sensitivity parameter of 0 represents null hypothesised levels of residual confounding and the extremes of − 1 and 1 represent maximum hypothesised levels of residual confounding. Grey zones represent 95% confidence limits of the estimated mediation effect across a range of hypothesised levels of residual confounding