Restricting evidence syntheses of interventions to English-language publications is a viable methodological shortcut for most medical topics: a systematic review

Abstract

Objectives: To assess the impact of restricting systematic reviews of conventional or alternative medical treatments or diagnostic tests to English-language publications.

Study design and setting: We systematically searched MEDLINE (Ovid), the Science Citation Index Expanded (Web of Science), and Current Contents Connect (Web of Science) up to April 24, 2020. Eligible methods studies assessed the impact of restricting systematic reviews to English-language publications on effect estimates and conclusions. Two reviewers independently screened the literature; one investigator performed the data extraction, a second investigator checked for completeness and accuracy. We synthesized the findings narratively.

Results: Eight methods studies (10 publications) met the inclusion criteria; none addressed language restrictions in diagnostic test accuracy reviews. The included studies analyzed nine to 147 meta-analyses and/or systematic reviews. The proportions of non-English-language publications ranged from 2% to 100%. Based on five methods studies, restricting literature searches or inclusion criteria to English-language publications led to a change in statistical significance in 23/259 meta-analyses (9%). Most commonly, the statistical significance was lost, but had no impact on the conclusions of systematic reviews.

Conclusion: Restricting systematic reviews to English-language publications appears to have little impact on the effect estimates and conclusions of systematic reviews.

Keywords: English-language restrictions; Language bias; Meta-research; Methods study.