Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Nov 24;7(1):208.
doi: 10.1186/s13643-018-0852-0.

Stakeholder involvement in systematic reviews: a scoping review

Alex Pollock  1 , Pauline Campbell  2 , Caroline Struthers  3 , Anneliese Synnot  4   5 , Jack Nunn  6 , Sophie Hill  4 , Heather Goodare  7 , Jacqui Morris  8 , Chris Watts  9 , Richard Morley  10
Affiliations

Stakeholder involvement in systematic reviews: a scoping review

Alex Pollock et al. Syst Rev. .

Abstract

Background: There is increasing recognition that it is good practice to involve stakeholders (meaning patients, the public, health professionals and others) in systematic reviews, but limited evidence about how best to do this. We aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews.

Methods: We carried out a scoping review, following a published protocol. We searched multiple electronic databases (2010-2016), using a stepwise searching approach, supplemented with hand searching. Two authors independently screened and discussed the first 500 abstracts and, after clarifying selection criteria, screened a further 500. Agreement on screening decisions was 97%, so screening was done by one reviewer only. Pre-planned data extraction was completed, and the comprehensiveness of the description of methods of involvement judged. Additional data extraction was completed for papers judged to have most comprehensive descriptions. Three stakeholder representatives were co-authors for this systematic review.

Results: We included 291 papers in which stakeholders were involved in a systematic review. Thirty percent involved patients and/or carers. Thirty-two percent were from the USA, 26% from the UK and 10% from Canada. Ten percent (32 reviews) were judged to provide a comprehensive description of methods of involving stakeholders. Sixty-nine percent (22/32) personally invited people to be involved; 22% (7/32) advertised opportunities to the general population. Eighty-one percent (26/32) had between 1 and 20 face-to-face meetings, with 83% of these holding ≤ 4 meetings. Meetings lasted 1 h to ½ day. Nineteen percent (6/32) used a Delphi method, most often involving three electronic rounds. Details of ethical approval were reported by 10/32. Expenses were reported to be paid to people involved in 8/32 systematic reviews.

Discussion/conclusion: We identified a relatively large number (291) of papers reporting stakeholder involvement in systematic reviews, but the quality of reporting was generally very poor. Information from a subset of papers judged to provide the best descriptions of stakeholder involvement in systematic reviews provide examples of different ways in which stakeholders have been involved in systematic reviews. These examples arguably currently provide the best available information to inform and guide decisions around the planning of stakeholder involvement within future systematic reviews. This evidence has been used to develop online learning resources.

Systematic review registration: The protocol for this systematic review was published on 21 April 2017. Publication reference: Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Watts C, Morley R: Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. Research Involvement and Engagement 2017, 3:9. https://doi.org/10.1186/s40900-017-0060-4 .

Keywords: Consumer; Involvement; Patient; Public; Scoping review; Stakeholder; Systematic review.

PubMed Disclaimer

Conflict of interest statement

Authors’ information

All authors are involved with Cochrane (an international not-for-profit which publishes systematic reviews).

Alex Pollock is a senior research fellow at the Scottish Government funded ‘Nursing Midwifery and Allied Health Professions Research Unit’ (NMAHP RU) and has been involved in systematic reviews of complex interventions for over 15 years. She is an associate editor of the Cochrane Stroke Group and an author on 12 Cochrane reviews. She has implemented innovative ways of involving stakeholders within Cochrane reviews.

Pauline Campbell is a research fellow at NMAHP RU, with particular responsibility for completing systematic reviews relating to nursing and healthcare interventions, including rapid reviews commissioned by the Scottish Government’s Chief Nursing Officer. She is an author on 7 Cochrane reviews and has involved stakeholders in reviews using a range of methods.

Caroline Struthers is the education and training manager for the EQUATOR Network, responsible for developing educational resources, courses and training materials for a wide range of users, including researchers, editors, peer reviewers and the general public. Previous roles have included Information Specialist, Managing Editor and Training Coordinator for Cochrane, and she was also manager of an NHS-funded public engagement project for the Cochrane Dementia and Cognitive Impairment Group. At the time of this work, she was a member of the Cochrane Consumer Network Executive.

Anneliese Synnot is a research fellow and editor with Cochrane Consumers and Communication. Her research interests include innovations in evidence synthesis, both in terms of novel methods to involve consumers and others in the review production and dissemination and innovative review types, such as Living Systematic Reviews.

Jack Nunn is a researcher in the Public Health Department in the Centre for Health Communication and Participation at La Trobe University. He has recently worked on projects with Cochrane Australia, the World Health Organisation and the Australian Department of Health. He has previously worked for the UK National Institute of Health Research and Macmillan Cancer Support (UK) where he led the development and implementation of an internationally recognised model for building partnerships between the public and researchers.

Sophie Hill is the Head of the Centre for Health Communication and Participation (www.latrobe.edu.au/chcp), at La Trobe University, a centre she has established from the foundation of the Cochrane Consumers and Communication Review Group (http://cccrg.cochrane.org/). The Centre has an applied focus, with three roles: coordinating the production and publication of evidence on interventions to communicate with people about health; innovative research on communication issues that have been neglected, such as multimorbidity; and a knowledge translation function, for getting evidence into practice and policy.

Heather Goodare is a Cochrane consumer reviewer for breast cancer and stroke; she was the first patient representative on the BMJ Editorial Board (1995–1999) and is a Life Fellow of the Royal Society of Medicine. Originally an academic book editor, Heather trained as a counsellor after her own experience of breast cancer and of a flawed research study (Chilvers et al. [64]) in which she was a patient (see Goodare [65]).

Chris Watts is the Learning and Support Officer for Cochrane Training. He works on a range of Cochrane learning projects including design and development of learning materials and pathways to support learners in a variety of Cochrane roles, particularly through online resources and initiatives for distance learners. Chris is a researcher by background and previously worked at the Royal College of Nursing in the UK, where he led a team of Research Analysts delivering evidence synthesis and evaluation projects supporting professional development and policy.

Richard Morley is the Consumer Coordinator for Cochrane, supporting consumer involvement in the production and dissemination of Cochrane evidence. He has extensive experience of public engagement and partnership working in the voluntary, public and education sectors.

Jacqui Morris is a Reader in Rehabilitation Research with a particular interest in how systematic review evidence can be implemented in allied health professionals practice. She is a co-author on several Cochrane reviews.

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
PRISMA flowchart
Fig. 2
Fig. 2
Bubble plot illustrating proportion of papers in which patients/carers were included, the stage of the review process at which people were involved and the comprehensiveness of the description of the method of involvement
See this image and copyright information in PMC

References

    1. Camden C, Shikako-Thomas K, Nguyen T, Graham E, Thomas A, Sprung J, Morris C, Russell DJ. Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disabil Rehabil. 2015;37:1390–1400. doi: 10.3109/09638288.2014.963705. - DOI - PubMed
    1. INVOLVE. Exploring the impact of public involvement on the quality of research: examples. Eastleigh: INVOLVE; 2013.
    1. Kreis J, Puhan MA, Schunemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16:323–337. doi: 10.1111/j.1369-7625.2011.00722.x. - DOI - PMC - PubMed
    1. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, Conway PH, Daudelin D, Morrato EH, Leslie LK. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27:985–991. doi: 10.1007/s11606-012-2037-1. - DOI - PMC - PubMed
    1. Heaton J, Day J, Britten N. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implement Sci. 2016;11:20. doi: 10.1186/s13012-016-0383-9. - DOI - PMC - PubMed

Publication types