Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Jun 22;13(1):84.
doi: 10.1186/s13012-018-0779-9.

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study

Christine Marquez  1 , Alekhya Mascarenhas Johnson  2 , Sabrina Jassemi  2 , Jamie Park  2 , Julia E Moore  2 , Caroline Blaine  3 , Gertrude Bourdon  4 , Mark Chignell  5 , Moriah E Ellen  6   7   8 , Jacques Fortin  9 , Ian D Graham  10   11 , Anne Hayes  12 , Jemila Hamid  2   13 , Brenda Hemmelgarn  14   15 , Michael Hillmer  7   12 , Bev Holmes  16   17 , Jayna Holroyd-Leduc  15   18 , Linda Hubert  19 , Brian Hutton  10 , Monika Kastner  7 , John N Lavis  8 , Karen Michell  20 , David Moher  10 , Mathieu Ouimet  21 , Laure Perrier  7 , Andrea Proctor  12 , Thomas Noseworthy  14 , Victoria Schuckel  22 , Sharlene Stayberg  23 , Marcello Tonelli  15 , Andrea C Tricco  2   24 , Sharon E Straus  2   5
Affiliations
Review

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study

Christine Marquez et al. Implement Sci. .

Abstract

Background: Systematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use.

Methods: A three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted with HCMs and PMs in four Canadian provinces to determine perceptions of a traditional systematic review format. In phase 2, systematic review format prototypes were created by HCMs and PMs via Conceptboard©. In phase 3, prototypes underwent usability testing by HCMs and PMs.

Results: Two hundred two participants (80 HCMs, 122 PMs) completed the phase 1 survey. Respondents reported that inadequate format (Mdn = 4; IQR = 4; range = 1-7) and content (Mdn = 4; IQR = 3; range = 1-7) influenced their use of systematic reviews. Most respondents (76%; n = 136/180) reported they would be more likely to use systematic reviews if the format was modified. Findings from 11 interviews (5 HCMs, 6 PMs) revealed that participants preferred systematic reviews of effects that were easy to access and read and provided more information on intervention effectiveness and less information on review methodology. The mean System Usability Scale (SUS) score was 55.7 (standard deviation [SD] 17.2) for the traditional format; a SUS score < 68 is below average usability. In phase 2, 14 HCMs and 20 PMs co-created prototypes, one for HCMs and one for PMs. HCMs preferred a traditional information order (i.e., methods, study flow diagram, forest plots) whereas PMs preferred an alternative order (i.e., background and key messages on one page; methods and limitations on another). In phase 3, the prototypes underwent usability testing with 5 HCMs and 7 PMs, 11 out of 12 participants co-created the prototypes (mean SUS score 86 [SD 9.3]).

Conclusions: HCMs and PMs co-created prototypes for systematic review of effects formats based on their needs. The prototypes will be compared to a traditional format in a randomized trial.

Keywords: Decision making; Evidence implementation; Health care managers; Integrated knowledge translation; Knowledge synthesis; Knowledge translation; Policy makers; Systematic reviews; Usability.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

Ethical approval was obtained from St. Michael’s Hospital Research Ethics Board (# 15-301) and the Interior Health Authority Research Ethics Board (# 2015-16-68-E). All participants provided informed consent.

Consent for publication

Not applicable.

Competing interests

SES is an editor for Implementation Science but was not involved in decisions related to this manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Study flow diagram
See this image and copyright information in PMC

References

    1. Straus SE, Tetroe J, Graham ID, Zwarenstein M, Bhattacharyya O, Shepperd S. Monitoring use of knowledge and evaluating outcomes. CMAJ. 2010;182(2):E94–E98. doi: 10.1503/cmaj.081335. - DOI - PMC - PubMed
    1. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–104. doi: 10.1016/S0140-6736(13)62329-6. - DOI - PubMed
    1. Brownlee S, Chalkidou K, Doust J, Elshaug AG, Glasziou P, Heath I, Nagpal S, Saini V, Srivastava D, Chalmers K, Korenstein D. Evidence for underuse of effective medical services around the world. Lancet. 2017;390(10090):169–177. doi: 10.1016/S0140-6736(16)32585-5. - DOI - PMC - PubMed
    1. Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001;39(8 Suppl 2):Ii46–Ii54. - PubMed
    1. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7:e1000326. doi: 10.1371/journal.pmed.1000326. - DOI - PMC - PubMed

Publication types

Grants and funding

LinkOut - more resources