Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2005 Apr 2;330(7494):753.
doi: 10.1136/bmj.38356.424606.8F. Epub 2005 Jan 28.

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors

An-Wen Chan  1 , Douglas G Altman
Affiliations

Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors

An-Wen Chan et al. BMJ. .

Abstract

Objective: To examine the extent and nature of outcome reporting bias in a broad cohort of published randomised trials.

Design: Retrospective review of publications and follow up survey of authors. Cohort All journal articles of randomised trials indexed in PubMed whose primary publication appeared in December 2000.

Main outcome measures: Prevalence of incompletely reported outcomes per trial; reasons for not reporting outcomes; association between completeness of reporting and statistical significance.

Results: 519 trials with 553 publications and 10,557 outcomes were identified. Survey responders (response rate 69%) provided information on unreported outcomes but were often unreliable--for 32% of those who denied the existence of such outcomes there was evidence to the contrary in their publications. On average, over 20% of the outcomes measured in a parallel group trial were incompletely reported. Within a trial, such outcomes had a higher odds of being statistically non-significant compared with fully reported outcomes (odds ratio 2.0 (95% confidence interval 1.6 to 2.7) for efficacy outcomes; 1.9 (1.1 to 3.5) for harm outcomes). The most commonly reported reasons for omitting efficacy outcomes included space constraints, lack of clinical importance, and lack of statistical significance.

Conclusions: Incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance. The medical literature therefore represents a selective and biased subset of study outcomes, and trial protocols should be made publicly available.

PubMed Disclaimer

Figures

Fig 1
Fig 1
Hierarchy of levels of outcome reporting (n=number of participants per group)
Fig 2
Fig 2
Identification of randomised trials from PubMed citations of studies published in December 2000
Fig 3
Fig 3
Odds ratios (black squares) with 95% confidence intervals for outcome reporting bias in randomised trials published in December 2000 and cited in PubMed. Size of the square reflects the weight of the trial in calculating the pooled odds ratio (diamond and dotted line)
See this image and copyright information in PMC

References

    1. Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract 2002;8: 353-9. - PubMed
    1. Tannock IF. False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst 1996;88: 206-7. - PubMed
    1. Streiner DL, Joffe R. The adequacy of reporting randomized, controlled trials in the evaluation of antidepressants. Can J Psychiatry 1998;43: 1026-30. - PubMed
    1. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291: 2457-65. - PubMed
    1. Chan AW, Krleža-Jerić K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171: 735-40. - PMC - PubMed

Publication types