Association Between Publication Characteristics and Treatment Effect Estimates: A Meta-epidemiologic Study

Abstract

Background: Evidence about the effect on meta-analysis results of including unpublished trials or those published in languages other than English is unclear or discordant.

Purpose: To compare treatment effects between published and unpublished randomized controlled trials (RCTs) and between trials published in English and other languages using a meta-epidemiologic approach.

Data sources: Cochrane reviews published between March 2011 and January 2017 and trial references cited in the reviews.

Study selection: RCTs included in meta-analyses of 3 or more trials with a binary efficacy outcome.

Data extraction: Trial characteristics were extracted by original review authors. A single reviewer assessed publication status and language, with quality assurance by another investigator.

Data synthesis: Among 5659 RCTs included in 698 meta-analyses, 5303 (93.7%) were published in journal articles and 356 (6.3%) were unpublished. Of journal articles, 92.6% (4910 of 5303) were published in English and 7.4% (393 of 5303) in another language. Treatment effects were larger in published than unpublished trials (combined ratio of odds ratios [ROR] for 174 meta-analyses, 0.90 [95% CI, 0.82 to 0.98]; I2 = 19.3%; τ2 = 0.0492). Treatment effects were also larger for trials published in a language other than English than in English (combined ROR for 147 meta-analyses, 0.86 [CI, 0.78 to 0.95]; I2 = 0%; τ2 = 0.0000).

Limitation: Reliance on the primary reference cited by review authors as the record of interest.

Conclusion: In meta-analyses, treatment effects were larger in published than unpublished trials and, for published trials, in those published in a language other than English than in English.

Primary funding source: Cochrane France.