Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2014 Nov 21:349:g6501.
doi: 10.1136/bmj.g6501.

Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews

Pooja Saini  1 , Yoon K Loke  2 , Carrol Gamble  3 , Douglas G Altman  4 , Paula R Williamson  3 , Jamie J Kirkham  5
Affiliations

Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews

Pooja Saini et al. BMJ. .

Abstract

Objective: To determine the extent and nature of selective non-reporting of harm outcomes in clinical studies that were eligible for inclusion in a cohort of systematic reviews.

Design: Cohort study of systematic reviews from two databases.

Setting: Outcome reporting bias in trials for harm outcomes (ORBIT II) in systematic reviews from the Cochrane Library and a separate cohort of systematic reviews of adverse events.

Participants: 92 systematic reviews of randomised controlled trials and non-randomised studies published in the Cochrane Library between issue 9, 2012 and issue 2, 2013 (Cochrane cohort) and 230 systematic reviews published between 1 January 2007 and 31 December 2011 in other publications, synthesising data on harm outcomes (adverse event cohort).

Methods: A 13 point classification system for missing outcome data on harm was developed and applied to the studies.

Results: 86% (79/92) of reviews in the Cochrane cohort did not include full data from the main harm outcome of interest of each review for all of the eligible studies included within that review; 76% (173/230) for the adverse event cohort. Overall, the single primary harm outcome was inadequately reported in 76% (705/931) of the studies included in the 92 reviews from the Cochrane cohort and not reported in 47% (4159/8837) of the 230 reviews in the adverse event cohort. In a sample of primary studies not reporting on the single primary harm outcome in the review, scrutiny of the study publication revealed that outcome reporting bias was suspected in nearly two thirds (63%, 248/393).

Conclusions: The number of reviews suspected of outcome reporting bias as a result of missing or partially reported harm related outcomes from at least one eligible study is high. The declaration of important harms and the quality of the reporting of harm outcomes must be improved in both primary studies and systematic reviews.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisation that might have an interest in the submitted work in the previous three years; YL is co-convenor of the Cochrane Adverse Effects Methods Group; however, the authors have no other relationships or activities that could appear to have influenced the submitted work.

Figures

None
Fig 1 Flow diagram for Cochrane cohort
None
Fig 2 Flow diagram for adverse event cohort
None
Fig 3 Assessment of studies within reviews (Cochrane cohort)
None
Fig 4 Assessment of studies within reviews (adverse event cohort)
None
Fig 5 Assessment of studies from sample of 50 reviews from each cohort
See this image and copyright information in PMC

Comment in

References

    1. Smyth R, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ 2011;342:c7153. - PMC - PubMed
    1. Hutton JL, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. Appl Stat 2000;49:359-70.
    1. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010;340:c365. - PubMed
    1. Chan A-W, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171:735-40. - PMC - PubMed
    1. Jonsson U, Alaie I, Parling T, Arnberg FK. Reporting of harms in randomized controlled trials of psychological interventions for mental and behavioural disorders: a review of current practice. Contemp Clin Trials 2014;38:1-8. - PubMed

Publication types

LinkOut - more resources