Uptake of methods to deal with publication bias in systematic reviews has increased over time, but there is still much scope for improvement

Abstract

Objective: To evaluate the measures taken to deal with publication bias across different categories of systematic reviews published in 2006 and to compare these with reviews published in 1996.

Study design and setting: PubMed was searched for systematic reviews published in 2006; 100 treatment effect, 50 diagnostic accuracy, 100 risk factor, and 50 gene-disease association reviews were randomly selected.

Results: The use of MEDLINE increased from 74% to 95%; checking references increased from 42% to 73%; use of Cochrane Library increased from 5% to 58%; and use of CINAHL increased from 8% in 1996 to 24% in treatment reviews, 20% in diagnostic reviews, 18% in risk factor reviews, and 0% in genetic reviews published in 2006. A 20% increase was observed for explicit searching of non-English-language studies in all reviews published in 2006. Efforts to search for unpublished studies increased to 61% from 35% in treatment reviews published in 1996. Twenty-six percent of the reviews used funnel plots or related methods to test for publication bias compared with less than 6% in earlier reviews.

Conclusion: Recent reviews show a significant improvement in the measures taken to prevent publication bias. However, few methods exist to deal with publication bias in the nonquantitative findings of systematic reviews.