Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Mar 25;8(3):e020869.
doi: 10.1136/bmjopen-2017-020869.

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer

Marius Goldkuhle  1 , Vikram M Narayan  2 , Aaron Weigl  1 , Philipp Dahm  2 , Nicole Skoetz  1
Affiliations

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer

Marius Goldkuhle et al. BMJ Open. .

Abstract

Objective: To compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.

Design: Methodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.

Data sources: Cancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.

Eligibility criteria for selecting studies: We included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.

Results: We included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)).

Conclusions: When comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.

Keywords: amstar; methodological systematic review; quality assessment.

PubMed Disclaimer

Conflict of interest statement

Competing interests: MG, AW and NS are part of Cochrane Haematological Malignancies, PD is part of Cochrane Urology.

Figures

Figure 1
Figure 1
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of Cochrane reviews. SR, systematic review.
Figure 2
Figure 2
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of high-impact journal SRs. SR, systematic review.
Figure 3
Figure 3
Forest plot comparing to what extent Cochrane reviews and SRs published in high-impact journals meet criteria for methodological quality. SR, systematic review
Figure 4
Figure 4
Distribution of Cochrane reviews and high-impact journal SRs by review question. SR, systematic review.
Figure 5
Figure 5
Distribution of Cochrane reviews and high-impact journal SRs by disease. SR, systematic review.
Figure 6
Figure 6
Distribution of Cochrane reviews and high-impact journal SRs by intervention. N/A, not applicable; SR, systematic review.
See this image and copyright information in PMC

References

    1. 2015 Journal Citation Reports®: Clarivate Analytics. 2017.
    1. Woolf S, Schünemann HJ, Eccles MP, et al. . Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implement Sci 2012;7:61 10.1186/1748-5908-7-61 - DOI - PMC - PubMed
    1. Mulrow CD. Rationale for systematic reviews. BMJ 1994;309:597–9. - PMC - PubMed
    1. Moher D, Tetzlaff J, Tricco AC, et al. . Epidemiology and reporting characteristics of systematic reviews. PLoS Med 2007;4:e78 10.1371/journal.pmed.0040078 - DOI - PMC - PubMed
    1. Page MJ, Shamseer L, Altman DG, et al. . Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study. PLoS Med 2016;13:e1002028 10.1371/journal.pmed.1002028 - DOI - PMC - PubMed

Publication types

LinkOut - more resources