Models for estimating the number of unpublished studies

Abstract

The possible existence of unreported studies can cast doubt on the conclusions of a meta-analytic summary of the literature, particularly if there is reason to believe that there is a publication bias against non-significant results. The present article proposes two general models that describe how the preponderance of published studies could report significant p-values even when testing a null hypothesis that is, in fact, true. Each such model allows one to estimate the number, N, of unpublished studies using the p-values reported in the published studies; the meta-analyst can then evaluate the plausibility of this estimated value of N, or related confidence bounds. Use of models of the kind suggested here allows meta-analysts to assess the problem of unpublished studies from various perspectives and thus can lead to greater understanding of, and confidence in, meta-analytic conclusions.