Exploring influences on evaluation practice: a case study of a national physical activity programme

Abstract

Background: Interventions to improve physical activity behaviour are a core part of public health policy and practice. It is essential that we evaluate these interventions and use the evidence to inform decisions to improve population health. Evaluation of 'real-world' interventions provide an opportunity to generate practice-relevant evidence, however these interventions are difficult to evaluate. Various guidelines have been developed to facilitate evaluation, but evidence about their effectiveness in practice is limited. To explore influences on evaluation practice in an applied context, we conducted a case study of Sport England's 'Get Healthy Get Active' (GHGA) programme. This was a national programme that funded 33 projects that were delivered and evaluated across England. The programme was chosen as it was designed to generate evidence on the role of sport in increasing physical activity and improving health. The study aimed to explore and appraise whether strategies intended to facilitate project evaluation, including funder requirements to use a standardised evaluation framework and specific data collection methods, were effective in generating evidence that enabled the programme to meet its aims.

Methods: We applied a collective case study design involving 35 semi-structured interviews, and documentary analysis of multiple sources of evidence from 23 physical activity projects funded by GHGA. We applied thematic and framework analysis. We developed a logic model and mapped actual outcomes against intended outcomes. A narrative synthesis is provided. We discuss implications for the effective commissioning and evaluation of public health interventions.

Results: We identified five main themes of influences on evaluation practices that can act as barriers and facilitators to good practice: programme and project design; evaluation design; partnerships; resources; and organisational structures and systems. These influences are context-specific and operate through a complex set of interactions.

Conclusion: Developing a better understanding of how influences on evaluation practice can act as facilitators or barriers is vital to help close current gaps in the evidence-based practice cycle. Critically, organisational structures and systems are needed to facilitate collaborative decision making; integration of projects and evaluation across partners organisations; transfer of knowldege and insights between stakeholders; and more rapid feedback and dissemination.

Keywords: Evaluation; Evidence-based public health; Influences on practice; Physical activity.

Fig. 1

Logic Model for the Get Healthy Get Active (GHGA) programme

Fig. 2

Pathway diagram of the Get Healthy Get Active (GHGA) programme. Notes: Round One was originally referred to as Get Healthy Get into Sport. Normal text shows external documents and influences on the programme e.g. Start Active Stay Active [40], Everybody Active Every Day [2], Bold text shows documents published or commissioned by Sport England and steps in the GHGA programme e.g. Sport England Strategy 2012–17 [41], Improving health through participation in sport [42], Get Healthy Get Active What we have learnt [34], Tackling Inactivity [43, 44]

Fig. 3

Evidence generated from the Get Healthy Get Active programme mapped against the intended outcomes. Notes:1Get Active Get Healthy, what we have learned so far [34], Tackling Inactivity [43], 2Design Principles [44], 3Sport England Evaluation Framework [45], 4Hertfordshire Evaluation Framework [49], 5Examples of publications include [–60]