Of mites and men: reference bias in narrative review articles: a systematic review

Abstract

Background: Citations in scientific articles may tend to favor the views presented. We studied whether there is such reference bias in narrative review articles that discuss interventions against house dust mites for people with asthma.

Design: Systematic review of reviews identified in a Medline search that expressed an opinion about the clinical effects of physical or chemical intervention methods.

Main outcome measure: Positive bias was judged to have occurred if the reference list contained a higher proportion of trial references with significant results than among all trials available to the authors (published 2 years or more prior to the review).

Results: Seventy reviews were included, of which 63 (90%) recommended physical interventions. Forty-six reviews had trial references, 4 of these only to chemical interventions. In the remaining 42 reviews, reference bias was detected (P=2 x 10-8). The most quoted trial had only 7 patients per group, its claimed significant result was probably erroneous, and it did not report a clinical outcome. Intervention recommendations were often based on nonrandomized evidence, and the most quoted nonrandomized controlled study had included only 10 patients per group but claimed very positive results.

Conclusion: The narrative review articles were severely biased, and their positive intervention recommendations are at variance with the systematic Cochrane Review on this topic and a recent very large trial of physical intervention, both of which failed to find an effect.