Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Nov 21;6(1):231.
doi: 10.1186/s13643-017-0617-1.

Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction

Carole Lunny  1 , Sue E Brennan  1 , Steve McDonald  1 , Joanne E McKenzie  2
Affiliations

Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction

Carole Lunny et al. Syst Rev. .

Abstract

Background: Overviews of systematic reviews attempt to systematically retrieve and summarise the results of multiple systematic reviews. Methods for conducting, interpreting and reporting overviews are in their infancy. To date, there has been no evidence map of the methods used in overviews, thus making it difficult to determine the gaps and priorities for methods research. Our objectives were to develop and populate a comprehensive framework of methods for conducting, interpreting and reporting overviews (stage I) and to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II).

Methods: We searched methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program) to identify eligible studies for both stages of this research. In stage I, cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews were used to develop and populate the framework of methods. Drafts and multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified and described studies evaluating overview methods and mapped these evaluations to the framework.

Results: In this paper, we present results for the four initial steps of conducting an overview: (a) specification of the purpose, objectives and scope, (b) specification of the eligibility criteria, (c) search methods and (d) data extraction. Twenty-nine studies mentioned or described methods relevant to one or more of these steps. In the developed framework, identified methods and approaches were grouped according to the steps an overview author would need to undertake. Fifteen studies evaluated identified methods, all of which mapped to the search methods step. These studies either reported the development and evaluation of a new search filter to retrieve systematic reviews or compared the performance of multiple filters.

Conclusion: Gaps in the evaluation of methods were found for the majority of steps in the framework. More empirical studies are needed to evaluate the methods outlined and provide a comprehensive evidence map. The framework is useful for planning these evaluations and for planning methods required to deal with challenges that arise when conducting an overview.

Keywords: Evaluation of methods; Evidence mapping; Evidence synthesis; Meta-review; Overview; Overview methods; Overviews of systematic reviews; Review of reviews; Systematic review methods; Umbrella review.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

JEM is an Associate Editor of Systematic Reviews and is a Guest Editor for the thematic series ‘Overviews of systematic reviews: development and evaluation of methods’, to which this paper was submitted. SEB is an Associate Editor of Systematic Reviews. Neither JEM nor SEB were involved in the peer-review or editorial decisions for this manuscript. CL and SM declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
Summary of the research reported in each paper
Fig. 2
Fig. 2
Stages in the development of an evidence map of overview methods
Fig. 3
Fig. 3
Flowchart of studies retrieved for both stages I and II. *The 42 stage I studies contributed to multiple steps
Fig. 4
Fig. 4
Flowchart of stage II studies of search filter evaluations
See this image and copyright information in PMC

References

    1. Becker LA, Oxman AD. In: Overviews of reviews. Cochrane Handbook for Systematic Reviews of Interventions. Higgins JPT, Green S, editors. Hoboken: John Wiley & Sons; 2008. pp. 607–631.
    1. Chen YF, Hemming K, Chilton PJ, Gupta KK, Altman DG, Lilford RJ. Scientific hypotheses can be tested by comparing the effects of one treatment over many diseases in a systematic review. J Clin Epidemiol. 2014;67(12):1309–1319. doi: 10.1016/j.jclinepi.2014.08.007. - DOI - PubMed
    1. Salanti G, Becker L, Caldwell D, Churchill R, Higgins J, Li T, Schmid C. Evolution of Cochrane intervention reviews and overviews of reviews to better accommodate comparisons among multiple interventions. In: Report from a meeting of the Cochrane comparing multiple interventions methods groups. Madrid: Cochrane Comparing Multiple Interventions Methods Groups; 2011.
    1. CMIMG C . Review Type & Methodological Considerations --background paper for the first part of the Paris CMIMG discussion. 2012.
    1. Lunny C, McKenzie JE, McDonald S. Retrieval of overviews of systematic reviews in MEDLINE was improved by the development of an objectively derived and validated search strategy. J Clin Epidemiol. 2016;74:107–118. doi: 10.1016/j.jclinepi.2015.12.002. - DOI - PubMed

Publication types

LinkOut - more resources