Publication bias is underreported in systematic reviews published in high-impact-factor journals: metaepidemiologic study

Abstract

Objectives: To examine how often a significant publication bias (PB) existed when the assessment of PB was not reported in systematic reviews.

Study design and setting: All systematic reviews with meta-analyses of interventions and risk/prognostic factors published in the general medical journals with the top 10 impact factors in 2011 and 2012 were included. The results regarding PB were extracted. When the assessment of PB was not reported, we examined the presence of PB using the Egger test and contour-enhanced funnel plot and the impact of unreported PB by regression-based method.

Results: Among all the identified 116 reviews, the assessment of PB was not reported in 36 reviews (31.0%), particularly in reviews without a comprehensive literature search. Of these 36 reviews, seven (19.4%) were found to have a significant PB. The original pooled results may have been overestimated by a median of 50.9% if corrected for PB. Among the 28 reviews with PB including both reviews that did or did not report the assessment of PB, seven reviews (25.0%) did not report the presence of PB.

Conclusion: Significant PB was underreported in systematic reviews published in high-impact-factor journals (eg, 19.4% of those that did not report assessment of PB had significant PB). Readers of systematic reviews should not assume that PB does not exist when not reported whereas researchers should report the results of assessments for PB.

Keywords: Funnel plot symmetry; Impact factor; Meta-analysis; Publication bias; Small-study effects; Systematic review.