Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2004 Sep 28;171(7):735-40.
doi: 10.1503/cmaj.1041086.

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research

An-Wen Chan  1 , Karmela Krleza-Jerić, Isabelle Schmid, Douglas G Altman
Affiliations
Meta-Analysis

Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research

An-Wen Chan et al. CMAJ. .

Abstract

Background: The reporting of outcomes within published randomized trials has previously been shown to be incomplete, biased and inconsistent with study protocols. We sought to determine whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review.

Methods: We compared protocols for randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications. Characteristics of reported and unreported outcomes were recorded from the protocols and publications. Incompletely reported outcomes were defined as those with insufficient data provided in publications for inclusion in meta-analyses. An overall odds ratio measuring the association between completeness of reporting and statistical significance was calculated stratified by trial. Finally, primary outcomes specified in trial protocols were compared with those reported in publications.

Results: We identified 48 trials with 68 publications and 1402 outcomes. The median number of participants per trial was 299, and 44% of the trials were published in general medical journals. A median of 31% (10th-90th percentile range 5%-67%) of outcomes measured to assess the efficacy of an intervention (efficacy outcomes) and 59% (0%-100%) of those measured to assess the harm of an intervention (harm outcomes) per trial were incompletely reported. Statistically significant efficacy outcomes had a higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95% confidence interval 1.5-5.0). Primary outcomes differed between protocols and publications for 40% of the trials.

Interpretation: Selective reporting of outcomes frequently occurs in publications of high-quality government-funded trials.

PubMed Disclaimer

Figures

None
Fig. 1: Hierarchy of the levels of outcome reporting. *See the online appendix (www.cmaj.ca/cgi/content/full/171/7/735/DC1) for descriptions of the amount of data required for meta-analyses of fully reported outcomes
None
Fig. 2: Identification of published randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998.
None
Fig. 3: Number of incompletely reported outcomes per trial, displayed according to total number of outcomes per trial, for outcomes measured to assess the efficacy of an intervention (top panel) and those measured to assess the harm of an intervention (bottom panel).
None
Fig. 4: Odds ratios (squares) and 95% confidence intervals for outcome reporting bias involving efficacy outcomes (30 trials, top panel) and harm outcomes (4 trials, bottom panel). The size of the square reflects the statistical weight of the trial in calculating the pooled odds ratio (represented by the diamond and dotted line).
See this image and copyright information in PMC

Comment in

References

    1. Chan AW, Haahr MT, Hróbjartsson A, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457-65. - PubMed
    1. Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. Publication and related biases [review]. Health Technol Assess 2000;4(10):1-115. - PubMed
    1. Whitehead A. Meta-analysis of controlled clinical trials. Chichester (UK): Wiley; 2002. p. 216.
    1. Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Egger M, Davey Smith G, Altman DG, editors. Systematic reviews in healthcare. Meta-analysis in context. 2nd ed. London (UK): BMJ Books; 2001. p. 285-312.
    1. McCormack K, Scott NW, Grant AM. Outcome reporting bias and individual patient data meta-analysis: a case study in surgery [abstract]. In: Abstracts for workshops and scientific sessions. 9th International Cochrane Colloquium; Lyon; 2001. p. 34-5.

Publication types