Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun;26(3):1255-1265.
doi: 10.1111/hex.13742. Epub 2023 Mar 21.

Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems

Julia Abelson  1 , Laura Tripp  1 , Maggie MacNeil  2 , Amy Lang  3 , Carol Fancott  4 , Rebecca Ganann  2 , Marisa Granieri  5 , Cathie Hofstetter  5 , Bernice King  5 , Betty-Lou Kristy  6 , Alies Maybee  5 , Maureen Smith  5 , Jeonghwa You  1 ; Evaluating Patient Engagement Working Group
Affiliations

Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems

Julia Abelson et al. Health Expect. 2023 Jun.

Abstract

Introduction: Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit-the Engage with Impact Toolkit.

Methods: The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design.

Results: The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1-5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit (www.evaluateengagement.ca) hosts the measurement framework and related evaluation supports.

Conclusion: The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work.

Patient contribution: Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.

Keywords: evaluation tools; family and caregiver engagement; impact evaluation; patient; public and patient engagement.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Toolkit development process.
Figure 2
Figure 2
Concept mapping results.
Figure 3
Figure 3
Phase 3 results.
Figure 4
Figure 4
The Engage with Impact Toolkit modules.
Figure 5
Figure 5
Screenshot of the online database.
See this image and copyright information in PMC

References

    1. Boivin A, Richards T, Forsythe L, et al. Evaluating patient and public involvement in research. BMJ. 2018;363:k5147. - PubMed
    1. Gilbert N, Cousins JB. Advancing patient engagement in health service improvement: what can the evaluation community offer? Can J Program Eval. 2017;32(2):202‐221.
    1. Forsythe L, Heckert A, Margolis MK, Schrandt S, Frank L. Methods and impact of engagement in research, from theory to practice and back again: early findings from the patient‐centered outcomes research institute. Qual Life Res. 2018;27(1):17‐31. - PMC - PubMed
    1. Abelson J, Humphrey A, Syrowatka A, Bidonde J, Judd M. Evaluating patient, family and public engagement in health services improvement and system redesign. Healthc Q. 2018;21(SP):61‐67. - PubMed
    1. Boivin A, L'Espérance A, Gauvin FP, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect. 2018;21:1075‐1084. - PMC - PubMed

Publication types