Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry

Abstract

Objectives: To present the contour-enhanced funnel plot as an aid to differentiating asymmetry due to publication bias from that due to other factors.

Study design and setting: An enhancement to the usual funnel plot is proposed that allows the statistical significance of study estimates to be considered. Contour lines indicating conventional milestones in levels of statistical significance (e.g., <0.01, <0.05, <0.1) are added to funnel plots.

Results: This contour overlay aids the interpretation of the funnel plot. For example, if studies appear to be missing in areas of statistical nonsignificance, then this adds credence to the possibility that the asymmetry is due to publication bias. Conversely, if the supposed missing studies are in areas of higher statistical significance, this would suggest the cause of the asymmetry may be more likely to be due to factors other than publication bias, such as variable study quality.

Conclusions: We believe this enhancement to funnel plots (i) is simple to implement, (ii) is widely applicable, (iii) greatly improves interpretability, and (iv) should be used routinely.