Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians
- PMID: 22972142
- PMCID: PMC11627148
- DOI: 10.1002/14651858.CD009401.pub2
Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians
Abstract
Background: Systematic reviews provide a transparent and robust summary of existing research. However, health system managers, national and local policy makers and healthcare professionals can face several obstacles when attempting to utilise this evidence. These include constraints operating within the health system, dealing with a large volume of research evidence and difficulties in adapting evidence from systematic reviews so that it is locally relevant. In an attempt to increase the use of systematic review evidence in decision-making a number of interventions have been developed. These include summaries of systematic review evidence that are designed to improve the accessibility of the findings of systematic reviews (often referred to as information products) and changes to organisational structures, such as employing specialist groups to synthesise the evidence to inform local decision-making.
Objectives: To identify and assess the effects of information products based on the findings of systematic review evidence and organisational supports and processes designed to support the uptake of systematic review evidence by health system managers, policy makers and healthcare professionals.
Search methods: We searched The Cochrane Library, MEDLINE, EMBASE, CINAHL, Web of Science, and Health Economic Evaluations Database. We also handsearched two journals (Implementation Science and Evidence and Policy), Cochrane Colloquium abstracts, websites of key organisations and reference lists of studies considered for inclusion. Searches were run from 1992 to March 2011 on all databases, an update search to March 2012 was run on MEDLINE only.
Selection criteria: Randomised controlled trials (RCTs), interrupted time-series (ITS) and controlled before-after studies (CBA) of interventions designed to aid the use of systematic reviews in healthcare decision-making were considered.
Data collection and analysis: Two review authors independently extracted the data and assessed the study quality. We extracted the median value across similar outcomes for each study and reported the range of values for each median value. We calculated the median of the two middlemost values if an even number of outcomes were reported.
Main results: We included eight studies evaluating the effectiveness of different interventions designed to support the uptake of systematic review evidence. The overall quality of the evidence was very low to moderate.Two cluster RCTs evaluated the effectiveness of multifaceted interventions, which contained access to systematic reviews relevant to reproductive health, to change obstetric care; the high baseline performance in some of the key clinical indicators limited the findings of these studies. There were no statistically significant effects on clinical practice for all but one of the clinical indicators in selected obstetric units in Thailand (median effect size 4.2%, range -11.2% to 18.2%) and none in Mexico (median effect size 3.5%, range 0.1% to 19.0%). In the second cluster RCT there were no statistically significant differences in selected obstetric units in the UK (median effect RR 0.92; range RR 0.57 to RR 1.10). One RCT evaluated the perceived understanding and ease of use of summary of findings tables in Cochrane Reviews. The median effect of the differences in responses for the acceptability of including summary of findings tables in Cochrane Reviews versus not including them was 16%, range 1% to 28%. One RCT evaluated the effect of an analgesic league table, derived from systematic review evidence, and there was no statistically significant effect on self-reported pain. Only one RCT evaluated an organisational intervention (which included a knowledge broker, access to a repository of systematic reviews and provision of tailored messages), and reported no statistically significant difference in evidence informed programme planning.Three interrupted time series studies evaluated the dissemination of printed bulletins based on evidence from systematic reviews. A statistically significant reduction in the rates of surgery for glue ear in children under 10 years (mean annual decline of -10.1%; 95% CI -7.9 to -12.3) and in children under 15 years (quarterly reduction -0.044; 95% CI -0.080 to -0.011) was reported. The distribution to general practitioners of a bulletin on the treatment of depression was associated with a statistically significant lower prescribing rate each quarter than that predicted by the rates of prescribing observed before the distribution of the bulletin (8.2%; P = 0.005).
Authors' conclusions: Mass mailing a printed bulletin which summarises systematic review evidence may improve evidence-based practice when there is a single clear message, if the change is relatively simple to accomplish, and there is a growing awareness by users of the evidence that a change in practice is required. If the intention is to develop awareness and knowledge of systematic review evidence, and the skills for implementing this evidence, a multifaceted intervention that addresses each of these aims may be required, though there is insufficient evidence to support this approach.
Conflict of interest statement
None
Figures
Update of
References
References to studies included in this review
Black 2002 {published data only}
Dobbins 2009 {published data only}
Gulmezoglu 2006 {published data only}
-
- Gulmezoglu AM, Langer A, Piaggio G, Lumbiganon P, Villar J, Grimshaw J. Cluster randomized trial of an active, multifaceted information dissemination intervention based on The WHO Reproductive Health Library to change obstetric practices. BJOG 2007 (published online 2006);114:16‐23.
Mason 1998/99 {published data only}
-
- Mason J, Freemantle N, Young P. The effect of the distribution of Effective Health Care bulletins on prescribing selective serotonin reuptake inhibitors in primary care. Health Trends 1998/99;30(4):120‐2.
Mason 2001 {published data only}
Rosenbaum 2010 {published data only}
-
- Rosenbaum SE, Glenton C, Oxman AD. Summary‐of‐findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. Journal of Clinical Epidemiology 2010;63(6):620‐6. - PubMed
Seers 2004 {published data only}
-
- Seers K, Crichton N, Carroll D, Richards S, Saunders T. Evidence‐based postoperative pain management in nursing: is a randomized‐controlled trial the most appropriate design?. Journal of Nursing Management 2004;12(3):183‐93. - PubMed
References to studies excluded from this review
Acelot 2007 {published data only}
Althabe 2008 {published data only}
-
- Althabe F, Bergel E, Cafferata ML, Gibbons L, Ciapponi A, Aleman A, et al. Strategies for improving the quality of health care in maternal and child health in low‐ and middle‐income countries: an overview of systematic reviews. Paediatric and Perinatal Epidemiology 2008;22 Suppl 1:42‐60. - PubMed
Booth 2000 {published data only}
-
- Booth A, Price C. Models for disseminating information on clinical effectiveness in health organisations: lessons from the Sheffield DICE Project. Journal of Clinical Excellence 2000;1:193‐200.
Bromirski 2011 {published data only}
-
- Bromirski BH, Cody JL, Coppin K, Hewson K, Richardson B. Evidence‐based practice day: an innovative educational opportunity. Western Journal of Nursing Research 2011;33:333‐44. - PubMed
Carroll 2006 {published data only}
-
- Carroll C, Cooke J, Booth A, Beverley C. Bridging the gap: the development of knowledge briefings at the health and social care interface. Health & Social Care in the Community 2006;14:491‐8. - PubMed
Ciliska 1999 {published data only}
-
- Ciliska D, Hayward S, Dobbins M, Brunton G, Underwood J. Transferring public‐health nursing research to health‐system planning: assessing the relevance and accessibility of systematic reviews. Canadian Journal of Nursing Research 1999;31(1):23‐36. - PubMed
Clarkson 2007 {published data only}
-
- Clarkson JE, Turner S, Grimshaw JM, Ramsay CR, Johnston M, Scott A, et al. Changing clinicians' behavior: a randomized controlled trial of fees and education. Journal of Dental Research 2008 Jul;87(7):640‐4. - PubMed
Costa 2009 {published data only}
-
- Costa ML, Cecatti JG, Milanez HM, Souza JP, Gulmezoglu M. Audit and feedback: effects on professional obstetrical practice and healthcare outcomes in a university hospital. Acta Obstetricia et Gynecologica Scandinavica 2009;88(7):793‐800. - PubMed
Debeck 2010 {published data only}
Dobbins 2001 {published data only}
-
- Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilization of systematic reviews. A study of public health decision makers. International Journal of Technology Assessment in Health Care 2001;17(2):203‐14. - PubMed
Dobbins 2001b {published data only}
-
- Dobbins M, Cockerill R, Barnsley J, Ciliska D. Factors of the innovation, organization, environment, and individual that predict the influence five systematic reviews had on public health decisions. International Journal of Technology Assessment in Health Care 2001b;17(4):467‐78. - PubMed
Dobbins 2004 {published data only}
-
- Dobbins M, Thomas H, O'Brien MA, Duggan M. Use of systematic reviews in the development of new provincial public health policies in Ontario. International Journal of Technology Assessment in Health Care 2004;20(4):399‐404. - PubMed
Dormuth 2004 {published data only}
Dunsford 2011 {published data only}
Dyson 2010 {published data only}
-
- Dyson L, Renfrew MJ, McFadden A, McCormick F, Herbert G, Thomas J. Policy and public health recommendations to promote the initiation and duration of breast‐feeding in developed country settings. Public Health Nutrition 2010;13:137‐44. - PubMed
Elliot 1997 {published data only}
-
- Elliott TE, Murray DM, Oken MM, Johnson KM, Braun BL, Elliott BA, et al. Improving cancer pain management in communities: main results from a randomized controlled trial. Journal of Pain & Symptom Management 1997;13:191‐203. - PubMed
Finnerup 2007 {published data only}
Fisher 2011 {published data only}
-
- Fisher RJ, Gaynor C, Kerr M, Langhorne P, Anderson C, Bautz‐Holter E, et al. A consensus on stroke: early supported discharge. Stroke 2011;42:1392‐7. - PubMed
Frantsve‐Hawley 2011 {published data only}
-
- Frantsve‐Hawley J, Jeske A. The American Dental Association's Center for Evidence‐Based Dentistry: a critical resource for 21st century dental practice. Texas Dental Journal 2011;128:201‐5. - PubMed
Grad 2011 {published data only}
Guiscfare 1995 {published data only}
-
- Guiscafre H, Martinez H, Reyes H, Perez‐Cuevas R, Castro R, Munoz O, Gutierrez G. From research to public health interventions 1. Impact of an educational strategy for physicians to improve treatment practices of common diseases. Archives of Medical Research 1995;26 (Spec No):S31‐9. - PubMed
Hayes 2001 {published data only}
-
- Hayes R, Bratzler D, Armour B, Moore l. Comparison of an enhanced versus written feedback model on the management of Medicare inpatients with venous thrombosis. Joint Commission Journal on Quality Improvement 2001;27(3):155‐68. - PubMed
Hutchinson 1997 {published data only}
-
- Hutchinson KA, Cotter SM, Dollman WB. QE‐Mail‐‐improving communication between the hospital and general practitioners. Australian Family Physician 1997;26 Suppl 2:99‐103. - PubMed
Jordans 2011 {published data only}
-
- Jordans MJD, Tol WA, Komproe IH. Mental health interventions for children in adversity: pilot‐testing a research strategy for treatment selection in low‐income settings stakeholder views of acceptability. Social Science & Medicine 2011;73:456‐66. - PubMed
Kar Purkayasha {published data only}
Lemelin 2001 {published data only}
McAlister 2009 {published data only}
Mettes 2007 {published data only}
-
- Mettes TG. A cluster randomised controlled trial in primary dental care based intervention to improve professional performance on routine oral examinations and the management of asymptomatic impacted third molars: study protocol. Implementation Science 2007;2(12):doi:10.1186/1748‐5908‐2‐12. - PMC - PubMed
Middleton 2009 {published data only}
Naughton 2009 {published data only}
-
- Naughton C, Feely J, Bennett K. A RCT evaluating the effectiveness and cost‐effectiveness of academic detailing versus postal prescribing feedback in changing GP antibiotic prescribing. Journal of Evaluation in Clinical Practice 2009;15:807‐12. - PubMed
Parry 1999 {published data only}
-
- Parry JM, Foy RC, Woodman CBJ, Deeks JJ, Watt I, Sheldon TA. A randomized controlled trial to assess the impact of a printed summary of research findings in general practice. Family Practice 1999;49:634‐8.
Pimlott 2003 {published data only}
Salbach 2011 {published data only}
-
- Salbach NM, Veinot P, Jaglal SB, Bayley M, Rolfe D. From continuing education to personal digital assistants: what do physical therapists need to support evidence‐based practice in stroke management?. Journal of Evaluation in Clinical Practice 2011;17:786‐93. - PubMed
Sales 2011 {published data only}
-
- Sales I, Amy DA. Avoidance of sulfonylureas in hospitalized patients at high risk for hypoglycemia: effectiveness of an email alert. http://www.pharmacotherapy.org/pdf/free/Pharm3110e_ACCP_Abstracts.pdf 2011;31(10):414e.
Spence 2011 {published data only}
-
- Spence K, Henderson‐Smart D. Closing the evidence‐practice gap for newborn pain using clinical networks. Journal of Paediatrics and Child Health 2011;47:92‐8. - PubMed
Suter 2011 {published data only}
Tanna 2011 {published data only}
-
- Tanna GV, Sood MM, Schiff J, Schwartz D, Naimark DM. Do e‐mail alerts of new research increase knowledge translation? A "Nephrology Now" randomized control trial. Academic Medicine 2011;86:132‐8. - PubMed
Thompson 2000 {published data only}
-
- Thompson RS, Rivara FP, Thompson DC, Barlow WE, Sugg NK, Maiuro RD, et al. Identification and management of domestic violence: a randomized trial. American Journal of Preventive Medicine 2000;19(4):253‐63. - PubMed
Thornton‐Jones 2002 {published data only}
-
- Thornton‐Jones H, Hampshaw S, Soltani H. Evidence, policy making and the art of the possible: a methodology applied to screening. Journal of Clinical Excellence 2002;4:9‐16.
Vollmar 2010 {published data only}
Williams 2009 {published data only}
-
- Williams GJ, Sureshkumar P, Wheeler D, Craig JC. Paediatrician's responses to an evidence summary about renal tract imaging tests in children after urinary tract infection. Archives of Disease in Childhood 2010;95(4):271‐5. - PubMed
References to ongoing studies
Additional references
Bastian 2010
Campbell 2002
-
- Campbell K, Waters E, O'Meara S, Kelly S, Summerbell C. Interventions for preventing obesity in children. Cochrane Databse of Systematic Reviews 2002;12:CD001871. - PubMed
Chambers 2011
Cook 1997
-
- Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Annals of Internal Medicine 1997;126:376‐80. - PubMed
Darzi 2008
-
- Darzi A. High quality care for all. Department of Health 2008.
Dobbins 2001a
-
- Dobbins M, Cockerill R, Barnsley J. Factors affecting the utilisation of systematic reviews. International Journal of Technology Assessment in Health Care 2001;17:203‐4. - PubMed
Elshaug 2009
-
- Elshaug AG, Moss JR, Littlejohns P, Karnon J, Merlin TL, Hiller JE. Identifying existing health care services that do not provide value for money. Medical Journal of Australia 2009;190:269‐73. - PubMed
EPOC 2010
-
- Cochrane Effective Practice and Organisation of Care Group. EPOC resources for review authors. http://epoc.cochrane.org/epoc‐resources‐review‐authors (accessed May 1, 2011).
Farmer 2008
-
- Farmer AP, Légaré F, Turcot L, Grimshaw J, Harvey E, McGowan J, Wolf FM. Printed educational materials: effects on professional practice and health care outcomes.. Cochrane Database of Systematic Reviews 2008;3:Art. No.: CD004398. DOI: 10.1002/14651858.CD004398.pub2. - PubMed
Flodgren 2011
Higgins 2008
-
- Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008]. Available from www.cochrane‐handbook.org.
Horsley 2011
Innvaer 2002
-
- Innvaer S, Vist G, Trommald M, Oxman A. Health policy‐makers' perceptions of their use of evidence: a systematic review. Journal of Health Services Reseach & Policy 2002;7(4):239‐44. - PubMed
Institute of Medicine 2011
-
- Institute of Medicine of National Academies. Clinical Practice Guidelines We Can Trust. Washington (DC): National Academies Press. National Academy of Sciences, 2011.
Jamtvedt 2006
-
- Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2006;2:Art. No.: CD000259. DOI: 10.1002/14651858.CD000259.pub2.. - PubMed
Lavis 2010
Lavis 2005
-
- Lavis J, Davies H, Oxman A, Denis JL, Golden‐Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy‐making. Journal of Health Services Research & Policy 2005;10 Suppl 1:35–48. - PubMed
Lavis 2009
Lefebvre 2009
-
- Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: Higgins JPT, Green S editor(s). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.2 (updated September 2009).. The Cochrane Collaboration, 2009.
Lundh 2009
-
- Lundh A, Knijnenburg SL, Jørgensen AW, Dalen EC, Kremer LCM. Quality of systematic reviews in pediatric oncology: a systematic review. Cancer Treatment Reviews 2009;35:645‐52. - PubMed
Moher 2007
Moseley 2009
-
- Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C. Cochrane reviews used more rigorous methods than non‐Cochrane reviews: survey of systematic reviews in physiotherapy. Journal of Clinical Epidemiology 2009;62(10):1021‐30. - PubMed
NHMRC 2011
-
- National Institute of Clinical Studies (NICS), National Health and Medical Research Council (NHMRC). What are the networks doing?. http://www.nhmrc.gov.au/nics/nics‐programs/volatile‐substance‐use/volati..., (accessed May 1, 2011).
NHS Evidence 2011
-
- National Health Service Excellence, National Institute for Clinical Evidence. Quality, Innovation, Productivity and Prevention (QIPP). http://www.evidence.nhs.uk/qipp, (accessed May 1, 2011).
NHS Evidence 2011b
-
- National Health Service Excellence, National Institute for Clinical Evidence. Evidence in Health and Social Care. http://www.evidence.nhs.uk/, (accessed May 1, 2011).
NIHR 2006
-
- National Institute for Health Research, National Health Service. NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs). http://www.nihr.ac.uk/infrastructure/Pages/infrastructure_clahrcs.aspx, (accessed May 1, 2011).
Peltz 1978
-
- Peltz DC. Some expanded perspectives on use of social science in public policy. In: Yinger JM, Cutler SJ editor(s). Major Social Issues: A Multidisciplinary View. New York Free Press, 1978:346‐357.
Perrier 2011
Perrier 2011a
Ram 2002
Walters S
-
- Walters S. Variation in baseline risk as an explanation of heterogenetiy in meta‐analysis. Statistics in Medicine 1997;16:2833‐900. - PubMed
Wen 2008
-
- Wen J, Ren Y, Wang L, Li Y, Liu Y, Zhou M, et al. The reporting quality of meta‐analyses improves: a random sampling study. Journal of Clinical Epidemiology 2008;61:770‐5. - PubMed
Wilson 2001
-
- Wilson PM, Watt IS, Hardman GF. Survey of medical directors’ views and use of the Cochrane Library. British Journal of Clinical Governance 2001;6(1):34‐9.
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
