Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2000 Jun 10;320(7249):1574-7.
doi: 10.1136/bmj.320.7249.1574.

Empirical assessment of effect of publication bias on meta-analyses

A J Sutton  1 , S J Duval, R L Tweedie, K R Abrams, D R Jones
Affiliations

Empirical assessment of effect of publication bias on meta-analyses

A J Sutton et al. BMJ. .

Abstract

Objective: To assess the effect of publication bias on the results and conclusions of systematic reviews and meta-analyses.

Design: Analysis of published meta-analyses by trim and fill method.

Studies: 48 reviews in Cochrane Database of Systematic Reviews that considered a binary endpoint and contained 10 or more individual studies.

Main outcome measures: Number of reviews with missing studies and effect on conclusions of meta-analyses.

Results: The trim and fill fixed effects analysis method estimated that 26 (54%) of reviews had missing studies and in 10 the number missing was significant. The corresponding figures with a random effects model were 23 (48%) and eight. In four cases, statistical inferences regarding the effect of the intervention were changed after the overall estimate for publication bias was adjusted for.

Conclusions: Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the conclusions. Nevertheless, researchers should check routinely whether conclusions of systematic reviews are robust to possible non-random selection mechanisms.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Typical funnel plot generated from 35 simulated studies (top) and same data with five missing studies showing a typical manifestation of publication bias (bottom)
Figure 2
Figure 2
Scatter plot of unadjusted versus adjusted odds ratios after filling with trim and fill method under random effects model. Dotted line is line of equality, and vertical distance away from this line indicates magnitude by which original pooled estimates have been reduced by adjusting for publication bias
See this image and copyright information in PMC

Comment in

References

    1. Song F, Easterwood A, Gilbody S, Duley L, Sutton, AJ. Publication bias. In: Stevens A, Abrams K, Brazier J, Fitzpatrick R, Lilford R, eds. Handbook of research methods for evidence-based health care—insights from the NHS HTA programme. London: Sage Publications (in press).
    1. Horton R. Medical editors trial amnesty. Lancet. 1997;350:756. - PubMed
    1. Easterbrook PJ. Directory of registries of clinical trials. Stat Med. 1992;11:345–423. - PubMed
    1. Begg CB. Publication bias. In: Cooper H, Hedges LV, editors. The handbook of research synthesis. New York: Russell Sage Foundation; 1994. pp. 399–409.
    1. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994;50:1088–1101. - PubMed