Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Feb 11;20(1):30.
doi: 10.1186/s12874-020-0898-2.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

Ian J Saldanha  1 , Kristina B Lindsley  2 , Sarah Money  3 , Hannah J Kimmel  4 , Bryant T Smith  4 , Kay Dickersin  5
Affiliations

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

Ian J Saldanha et al. BMC Med Res Methodol. .

Abstract

Background: There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.

Methods: We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.

Results: We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews).

Conclusions: Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Keywords: Clinical trials; Core outcome sets; Loss of information; Meta-analysis; Outcomes; Systematic review.

PubMed Disclaimer

Conflict of interest statement

Drs. Saldanha and Dickersin and Ms. Lindsley and Money declare affiliation with Cochrane Eyes and Vision during conduct of the work related to this manuscript. Ms. Kimmel and Mr. Smith were research assistants on this study. All of the reviews examined in this manuscript were produced by Cochrane Eyes and Vision. No other disclosures are reported.

Figures

Fig. 1
Fig. 1
Conduct of meta-analyses for the main outcome domain
Fig. 2
Fig. 2
Percentage of studies included in the review that were incorporated into a meta-analysis for any outcome (blue line) and for the review’s main outcome (red bars) Notes: This Figure excludes the 50 systematic reviews in whom a meta-analysis was not possible: 35 systematic reviews that each included 0 studies (i.e., “empty reviews”) and 15 systematic reviews included that each included only 1 study. When the blue line is non-0 but the red bars are 0, it implies that the systematic review did not conduct a meta-analysis for the main outcome, but did so for ≥1 of the remaining outcomes
See this image and copyright information in PMC

References

    1. Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press; 2011. - PubMed
    1. Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods. 2010;1(2):97–111. doi: 10.1002/jrsm.12. - DOI - PubMed
    1. Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. Version 1.02 ed. London: Cochrane; 2016.
    1. Saldanha IJ, Le JT, Solomon SD, Repka MX, Akpek EK, Li T. Choosing Core outcomes for use in clinical trials in ophthalmology: perspectives from three ophthalmology outcomes working groups. Ophthalmology. 2019;126(1):6–9. doi: 10.1016/j.ophtha.2018.09.008. - DOI - PMC - PubMed
    1. Meinert CL. Clinical trials dictionary: terminology and usage recommendation. 2nd ed Hoboken, NJ: Wiley; 2012.

Publication types

MeSH terms

LinkOut - more resources