Applying pragmatic approaches to complex program evaluation: A case study of implementation of the New South Wales Get Healthy at Work program

Abstract

Issue addressed: Complex health promotion programs, which can have multilevels of implementation and multi-components with nonlinear causal pathways, present many evaluation challenges. Traditional evaluation methods often fail to account for the complexity inherent in assessing these programs. In real-world settings, evaluations of complex programs are often beset by additional constraints of limited budgets and short timeframes. Determining whether a complex program is successful and how a program worked requires evaluators of complex programs to adopt a level of pragmatism.

Methods: This paper describes a pragmatic evaluation approach used to evaluate the Get Healthy at Work workplace health promotion program, implemented in New South Wales, Australia. Using the program as a case study, we describe some key principles for applying a pragmatic evaluation approach and use these principles to develop an appropriate evaluation strategy.

Results: The evaluation includes multiple research methods to assess program outputs and implementation; and identify emergent program impacts, within constrained resources. The evaluation was guided by epistemological flexibility, methodological comprehensiveness and operational practicality.

Conclusion: Health promotion programs, such as state-wide obesity prevention programs, require appropriate evaluation methods which address their inherent complexity amidst the real-world evaluation constraints, and focuses on the essential evaluation needs.

So what: The main complex program evaluation principles are applicable to other multilevel health promotion programs, challenged by methodological and practical or political constraints.

Keywords: evaluation methods; health promotion theory; obesity; program evaluation; workplaces.