Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2008 Jan 23;2008(1):MR000023.
doi: 10.1002/14651858.MR000023.pub3.

When and how to update systematic reviews

D Moher  1 , A Tsertsvadze, A C Tricco, M Eccles, J Grimshaw, M Sampson, N Barrowman
Affiliations

When and how to update systematic reviews

D Moher et al. Cochrane Database Syst Rev. .

Abstract

Background: Systematic reviews are most helpful if they are up-to-date. We did a systematic review of strategies and methods describing when and how to update systematic reviews.

Objectives: To identify, describe and assess strategies and methods addressing: 1) when to update systematic reviews and 2) how to update systematic reviews.

Search strategy: We searched MEDLINE (1966 to December 2005), PsycINFO, the Cochrane Methodology Register (Issue 1, 2006), and hand searched the 2005 Cochrane Colloquium proceedings.

Selection criteria: We included methodology reports, updated systematic reviews, commentaries, editorials, or other short reports describing the development, use, or comparison of strategies and methods for determining the need for updating or updating systematic reviews in healthcare.

Data collection and analysis: We abstracted information from each included report using a 15-item questionnaire. The strategies and methods for updating systematic reviews were assessed and compared descriptively with respect to their usefulness, comprehensiveness, advantages, and disadvantages.

Main results: Four updating strategies, one technique, and two statistical methods were identified. Three strategies addressed steps for updating and one strategy presented a model for assessing the need to update. One technique discussed the use of the "entry date" field in bibliographic searching. Statistical methods were cumulative meta-analysis and predicting when meta-analyses are outdated.

Authors' conclusions: Little research has been conducted on when and how to update systematic reviews and the feasibility and efficiency of the identified approaches is uncertain. These shortcomings should be addressed in future research.

PubMed Disclaimer

Conflict of interest statement

None.

Update of

  • doi: 10.1002/14651858.MR000023.pub2

References

References to studies included in this review

Barrowman 2003 {published data only}
    1. Barrowman NJ, Fang M, Sampson M, Moher D. Identifying null meta‐analyses that are ripe for updating.. BMC Medical Research Methodology 2003;3(1):13. [MEDLINE: ] - PMC - PubMed
Baum 1981 {published data only}
    1. Baum ML, Anish DS, Chalmers TC, Sacks HS, Smith Jr. H, Fagerstrom RM. A survey of clinical trials of antibiotic prophylaxis in colon surgery: evidence against further use of no‐treatment controls.. The New England Journal of Medicine 1981;305:795‐799. [MEDLINE: ] - PubMed
Berkey 1996 {published data only}
    1. Berkey CS, Mosteller F, Lau J, Antman EM. Uncertainty of the time of first significance in random effects cumulative meta‐analysis.. Controlled Clinical Trials 1996;17(5):357‐371. [MEDLINE: ] - PubMed
Chalmers 1993 {published data only}
    1. Chalmers I, Enkin M, Keirse MJ. Preparing and updating systematic reviews of randomized controlled trials of health care.. Milbank Quarterly 1993;71(3):411‐437. [MEDLINE: ] - PubMed
Higgins 2005 {published data only}
    1. Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5 [updated May 2005].. www.cochrane.org/resources/handbook/hbook.htm (accessed 25 September 2005)..
Ioannidis 1999 {published data only}
    1. Ioannidis JP, Contopolous‐Ioannidis DG, Lau J. Recursive cumulative meta‐analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data.. Journal of Clinical Epidemiology 1999;52:281‐291. [MEDLINE: ] - PubMed
Ioannidis 2001 {published data only}
    1. Ioannidis JP, Lau J. Evolution of treatment effects over time: Empirical insight from recursive cumulative meta‐analyses.. Proceedings of the National Academy of Sciences of the United States of America. 2001; Vol. 98, issue 3:831‐836. - PMC - PubMed
Lau 1992 {published data only}
    1. Lau J, Antman EM, Jimenez‐Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta‐analysis of therapeutic trials for myocardial infarction.. The New England Journal of Medicine 1992;327(4):248‐254. [MEDLINE: ] - PubMed
Lau 1995 {published data only}
    1. Lau J, Schmid CH, Chalmers TC. Cumulative meta‐analysis of clinical trials builds evidence for exemplary medical care.. Journal of Clinical Epidemiology 1995;48:45‐57. [MEDLINE: ] - PubMed
Lutje 2005 {published data only}
    1. Lutje V, MacLehose H, Garner P. Editorial strategy for revising Cochrane reviews: does it help?. 13th Cochrane Colloquium. 2005:153.
Mullen 2001 {published data only}
    1. Mullen B, Muerllereile P, Bryant B. Cumulative meta‐analysis: a consideration of indicators of sufficiency and stability.. Personality and Social Psychology Bulletin 2001;27:1450‐1462.
Pogue 1997 {published data only}
    1. Pogue JM, Yusuf S. Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta‐analysis.. Controlled Clinical Trials 1997;18(6):580‐593. [MEDLINE: ] - PubMed
Sutton 2006 {published data only}
    1. Sutton AJ, Gamble C, Donehue S. Evaluating two statistical methods to predict when a meta‐analysis will gain sufficient power to change the conclusions of a Cochrane review.. XIV Cochrane Colloquium, Dublin, Ireland.. 2006.
Weller 1998 {published data only}
    1. Weller D. Managing and updating the evidence.. Evidence‐based Health Advice Workshop; November 4‐5, 1998. Adelaide, Australia. 1998:1‐8.

References to studies excluded from this review

Chapman 2002a {published data only}
    1. Chapman A, Middleton P, Maddern G. Early updates of systematic reviews ‐ a waste of resources?. 4th Symposium on Systematic Reviews: Pushing the Boundaries, July 2002. 2002.
Counsell 1994 {published data only}
    1. Counsell CE, Fraser H, Sandercock PA. Archie Cochrane's challenge: can periodically updated reviews of all randomised controlled trials relevant to neurology and neurosurgery be produced?. Journal of Neurology, Neurosurgery, and Psychiatry 1994;57(5):529‐533. [MEDLINE: ] - PMC - PubMed
Eccles 2002 {published data only}
    1. Eccles M, Rousseau N, Freemantle N. Updating evidence‐based clinical guidelines.. Journal of Health Services Research and Policy 2002;7(2):98‐103. [MEDLINE: ] - PubMed
Enkin 1996 {published data only}
    1. Enkin M, Hetherington J. Collecting the evidence systematically. Ensuring that it is complete and up‐to‐date.. International Journal of Technology Assessment in Health Care 1996;12(2):276‐279. [MEDLINE: ] - PubMed
French 2004 {published data only}
    1. French S, Green S, McDonald S. Should we update Cochrane reviews every two years? A methodological study.. 12th Cochrane Colloquium: Bridging the Gaps; 2004 Oct 2‐6. 2004:86‐87.
McCormack 2004 {published data only}
    1. McCormack K, Grant A, Scott N, EU Hernia TC. Value of updating a systematic review in surgery using individual patient data.. British Journal of Surgery 2004;91(4):495‐499. [MEDLINE: ] - PubMed
Mowatt 1998 {published data only}
    1. Mowatt G, Bower DJ, Brebner JA, Cairns JA, Grant AM, McKee L. When is the 'right' time to initiate as assessment of a health technology.. International Journal of Health Technology Assessment in Health Care 1998;14(2):372‐386. [MEDLINE: ] - PubMed
Orwin 1983 {published data only}
    1. Orwin RG. A fail‐safe N for effect size in meta‐analysis.. Journal of Educational Statistics 1983;8(2):157‐159.
Stead 2001a {published data only}
    1. Stead LF, Lancaster T, Silagy CA. Updating a systematic review ‐ what difference did it make? Case study of nicotine replacement therapy.. BMC Medical Research Methodology 2001;1:10. [MEDLINE: ] - PMC - PubMed
Urquhart 1996 {published data only}
    1. Urquhart C, Hepworth J. The value of information supplied to clinicians by health libraries: devising an outcomes‐based assessment of the contribution of libraries to clinical decision‐making.. Health Libraries Review 1995;12(3):201‐213. [MEDLINE: ] - PubMed
van Tulder 2003 {published data only}
    1. Tulder M, Furlan A, Bombardier C, Bouter L, Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group.. Spine 2003;28(12):1290‐1299. [MEDLINE: ] - PubMed

Additional references

AGREE 2005
    1. The Appraisal of Guidelines Research and Evaluation (the AGREE collaboration). Appraisal of Guidelines Research and Evaluation.. http://www.agreecollaboration.org Accessed on 29th September, 2005..
Alderson 1997
    1. Alderson P, Roberts I. Corticosteroids in acute traumatic brain injury: a systematic review of randomised trials.. British Medical Journal 1997;314:1855‐1859. [MEDLINE: ] - PMC - PubMed
Antman 1992
    1. Antman EM, Lau J, Kupelnick B. A comparison of results of meta‐analyses of randomized control trials and recommendations of clinical experts.. JAMA 1992;268:240‐248. [MEDLINE: ] - PubMed
Atkins 2005
    1. Atkins D, Fink K, Slutsky J. Better information for better health care: the Evidence‐based Practice Center Program and the Agency for Healthcare Research and Quality.. Annals of Internal Medicine 2005;142:1035‐1041. [MEDLINE: ] - PubMed
Chalmers 1986
    1. Chalmers I. Electronic publications for updating controlled trial reviews.. The Lancet 1986;2:287.
Chalmers 1991
    1. Chalmers TC. Problems induced by meta‐analysis.. Statistics in Medicine 1991;10:971‐980. [MEDLINE: ] - PubMed
Chalmers 1994
    1. Chalmers I, Haynes B. Systematic reviews: reporting, updating, and correcting systematic reviews of the effects of health care.. British Medical Journal 1994;309:862‐865. [MEDLINE: ] - PMC - PubMed
Chapman 2002b
    1. Chapman A, Middleton P, Maddern G. Early updates of systematic reviews ‐ a waste of resources?. Pushing the Boundaries. Proceedings of the 4th Symposium on Systematic Reviews: July, 2002. Oxford, England. 2002.
Claxton 2004
    1. Claxton K, Ginnelly L, Sculpher M, Philips Z, Palmer S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme.. Health Technology Assessment 2004;8:31. [MEDLINE: ] - PubMed
Clinton 1994
    1. Clinton JJ. Health Care policy and Research. Process for determining need for updates of clinical practice guidelines.. Federal Register 1994;59(79):19723‐19725. [MEDLINE: ] - PubMed
Cook 1997a
    1. Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions.. Annals of Internal Medicine 1997;126(5):376‐380. [MEDLINE: ] - PubMed
Cook 1997b
    1. Cook DJ, Greengol NL, Ellrodt AG, Weingarten SR. The relation between systematic reviews and practice guidelines.. Annals of Internal Medicine 1997;127(3):210‐216. [MEDLINE: ] - PubMed
CRASH 2004
    1. CRASH trial collaborators. Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo‐controlled trial.. The Lancet 2004;364:1321‐1328. [MEDLINE: ] - PubMed
Dillon 2005
    1. Dillon A, National Institute for Health and Clinical Excellence (NICE). Updating process and methods guides. National Institute for Health and Clinical Excellence (NICE). website (http:// www.nice.org.uk/pdf/GDM_Chapter15.pdf) Accessed on 10th December, 2005..
Eccles 2001
    1. Eccles M, Freemantle N, Mason J. Using systematic reviews in clinical guideline development.. G. Davey Smith, M. Egger, D. Altman, editor(s). Systematic reviews in health care. Meta‐analysis in context.. 2nd Edition. British Medical Journal, 2001:400‐409.
Egger 2001
    1. Egger M, Smith GD, Altman DG. Systematic reviews in health care: meta‐analysis in context.. 2nd Edition. British Medical Journal Publishing Group, 2001.
Fergusson 2005
    1. Fergusson D, Glass KC, Hutton B, Shapiro S. Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding?. Clinical Trials 2005;2:218‐232. [MEDLINE: ] - PubMed
French 2005
    1. French S, McDonald S, McKenzie JE, Green S. Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?. BMC Medical Research Methodology 2005;5:33. [MEDLINE: ] - PMC - PubMed
Friedman 1998
    1. Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials. 3rd Edition. Springer‐Verlag, 1998.
Gartlehner 2004
    1. Gartlehner G, West SL, Lohr KN, Kahwati L, Johnson JG, Harris RP, Whitener L. Assessing the need to update prevention guidelines: a comparison of two methods.. International Journal of Quality in Health Care 2004;16(5):399‐406. [MEDLINE: ] - PubMed
Higgins 1999
    1. Higgins JPT. Prevalence and problems of updated reviews: a survey and discussion.. Beyond the Basics. Proceedings of the 2nd Symposium on Systematic Reviews: 1999 Oxford, England. 1999.
Hopewell 2001
    1. Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for results of clinical trials. Cochrane Database of Systematic Reviews 2007, Issue 2. [DOI: 10.1002/14651858.MR000011.pub2] - DOI - PMC - PubMed
Ioannidis 1998
    1. Ioannidis JP, Collier AC, Cooper DA, Corey L. Clinical efficacy of high‐dose acyclovir in patients with human immunodeficiency virus infection.. Journal of infectious Diseases 1998;178:349‐359. [MEDLINE: ] - PubMed
Jadad 1998
    1. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher D. Methodology and reports of systematic reviews and meta‐analyses. A comparison of Cochrane reviews with articles published in paper‐based journals.. Journal of American medical Association 1998;280:278‐280. [MEDLINE: ] - PubMed
Johnston 2003
    1. Johnston ME, Brouwers MC, Browman GP. Keeping cancer guidelines current: results of a comprehensive prospective literature monitoring strategy for twenty clinical practice guidelines.. International Journal of Technology Assessment in Health Care 2003;19(4):646‐655. [MEDLINE: ] - PubMed
Koch 2002
    1. Koch G. Are Cochrane Reviews updated regularly or not?. Proceedings of the 10th Cochrane Colloquium: 31 July ‐ 3 August.. 2002.
Moher 2005
    1. Moher D, Tetzlaff J, Tricco A, Sampson M, Altman D. Epidemiology and reporting characteristics of systematic reviews. Proceedings of the 4th Canadian Cochrane Symposium. 2005. - PMC - PubMed
Moher 2006
    1. Moher D, Tsertsvadze A. Systematic reviews: when is an update an update?. The Lancet 2006;367:881‐883. [MEDLINE: ] - PubMed
Moher 2007
    1. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman D. Epidemiology and reporting characteristics of systematic reviews.. PLoS Medicine 2007;4(3):e78. [MEDLINE: ] - PMC - PubMed
Montori 2005
    1. Montori VM, Devereaux PJ, Adhikari NKJ, Burns KEA, Eggert CH, Briel M. Randomized trials stopped early for benefit: a systematic review.. Journal of American Medical Association 2005;294:2203‐2209. [MEDLINE: ] - PubMed
Mulrow 1994
    1. Mulrow CD. Systematic reviews: rationale for systematic reviews. British Medical Journal 1994;309:597‐599. [MEDLINE: ] - PMC - PubMed
Shekelle 2001a
    1. Shekelle PG, Eccles MP, Grimshaw JM, Woolf SH. When should clinical guidelines be updated?. British Medical Journal 2001;323:155‐157. [MEDLINE: ] - PMC - PubMed
Shekelle 2001b
    1. Shekelle PG, Ortiz E, Rhodes S, Morton SC, Eccles MP, Grimshaw JM, Woolf SH. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated?. Journal of American Medical Association 2001;286(12):1461‐1467. [MEDLINE: ] - PubMed
Song 2000
    1. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication and related biases.. Health Technology Assessment 2000;4(10):1‐115. [MEDLINE: ] - PubMed
Stead 2001b
    1. Stead LF, Lancaster T, Silagy CA. Updating a systematic review ‐ what difference did it make? Case study of nicotine replacement therapy.. BMC Medical Research Methodology 2001;1:10. [MEDLINE: ] - PMC - PubMed
Sutton 1998
    1. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Systematic reviews of trials and other studies.. Health Technology Assessment 1998;2(19):1‐276. [MEDLINE: ] - PubMed
Sutton 2002
    1. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for meta‐analysis in medical research.. John Wiley & Sons Ltd,, 2002.
Sutton Unpublished
    1. Alex J. Sutton, Sarah Donehue, Paul Garner, Carol Gamble, Alison Donald Sutton AJ, Donehue S, Garner P, Gamble C, Donald A. Predicting the conclusions of an updated meta‐analysis for the purposes of prioritising the updating of systematic reviews.. Unpublished, 2007.
Swets 1988
    1. Swets JA. Measuring the accuracy of diagnostic systems.. Science 1988;240:1285‐93. [MEDLINE: ] - PubMed
Webster's 1996
    1. Merriam‐Webster. Merriam‐Webster's Collegiate Dictionary.. 10th edition. Springfield, Massachusetts: Merriam‐Webster 1996.
Wiebe 2006
    1. Wiebe N, Vandermeer B, Platt RW, Klassen TP, Moher D, Barrowman NJ. A systematic review identifies a lack of standardization in methods for handling missing variance data.. Journal of Clinical Epidemiology 2006;59:342‐353. [MEDLINE: ] - PubMed
Zarnke 2000
    1. Zarnke KB, Campbell NRC, McAlister FA, Levine M. A novel process for updating recommendations for managing hypertension.. Can J Cardiol 2000;16(9):1094‐1102. [MEDLINE: ] - PubMed

References to other published versions of this review

Moher D 2007
    1. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, Barrowman N. A Systematic Review Identified few Methods and Strategies describing when and how to update systematic reviews. Journal of Clinical Epidemiology 2007;60:1095‐1104. - PubMed

Publication types

LinkOut - more resources