Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2010 May 12:11:52.
doi: 10.1186/1745-6215-11-52.

Assessing the potential for outcome reporting bias in a review: a tutorial

Kerry Dwan  1 , Carrol Gamble, Ruwanthi Kolamunnage-Dona, Shabana Mohammed, Colin Powell, Paula R Williamson
Affiliations

Assessing the potential for outcome reporting bias in a review: a tutorial

Kerry Dwan et al. Trials. .

Abstract

Background: Outcome reporting bias (ORB) occurs when variables are selected for publication based on their results. This can impact upon the results of a meta-analysis, biasing the pooled treatment effect estimate.The aim of this paper is to show how to assess a systematic review and corresponding trial reports for ORB using an example review of intravenous and nebulised magnesium in the treatment of asthma.

Methods: The review was assessed for ORB by 1) checking the reasons, when available, for excluding studies to ensure that no studies were excluded because they did not report the outcomes of interest in the review; 2) assessing the eligible studies as to whether the review outcomes of interest were reported. Each study was classified using a system developed in the ORBIT (Outcome Reporting Bias In Trials) project to indicate whether ORB was suspected and a reason for the suspicion. Authors of trials that did not report the outcomes of interest were contacted for information. A sensitivity analysis was performed to assess the robustness of the conclusions of the review to this potential source of bias.

Results: Twenty-four studies were included in the review; two studies had been excluded for not reporting either of the two outcomes of interest. Six included studies did not report hospital admission and two did not report pulmonary function. There was high suspicion of outcome reporting bias in four studies. Results from the sensitivity analysis indicate that review conclusions were not overturned.

Conclusion: This paper demonstrates, with the example of the magnesium review, how to assess a review for outcome reporting bias. A review should not exclude studies if they have not reported the outcomes of interest and should consider the potential for outcome reporting bias in all included studies.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Plot of the treatment effect estimate using the bound for maximum bias sensitivity analysis for hospital admission for children.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Jones R. Strength of evidence in qualitative research. J Clin Epidemiol. 2007;60(4):321–3. doi: 10.1016/j.jclinepi.2006.06.001. - DOI - PubMed
    1. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.0. The Cochrane Collaboration; 2008.
    1. Egger M, Smith Davey G, Altman DG. Systematic Reviews in Health Care. Meta-analysis in context. Second. BMJ Publishing Group; 2001.
    1. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. PLoS ONE. 2008;3(8):e3081. doi: 10.1371/journal.pone.0003081. - DOI - PMC - PubMed
    1. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009. p. MR000006. - PMC - PubMed

Publication types