Why publishing everything is more effective than selective publishing of statistically significant results

Abstract

Background: De Winter and Happee examined whether science based on selective publishing of significant results may be effective in accurate estimation of population effects, and whether this is even more effective than a science in which all results are published (i.e., a science without publication bias). Based on their simulation study they concluded that "selective publishing yields a more accurate meta-analytic estimation of the true effect than publishing everything, (and that) publishing nonreplicable results while placing null results in the file drawer can be beneficial for the scientific collective" (p.4).

Methods and findings: Using their scenario with a small to medium population effect size, we show that publishing everything is more effective for the scientific collective than selective publishing of significant results. Additionally, we examined a scenario with a null effect, which provides a more dramatic illustration of the superiority of publishing everything over selective publishing.

Conclusion: Publishing everything is more effective than only reporting significant outcomes.

Figure 1. Mean (solid lines) and mean plus/minus one meta-analytic standard error (dashed lines) of estimate obtained by a random-effect meta-analysis as a function of the number of publications for both the publishing everything and selective publishing approaches.