Retrospective cohort study highlighted outcome reporting bias in UK publicly funded trials

Abstract

Objective: To assess outcome reporting bias and dissemination bias in trials funded by the National Health System (NHS) Health Technology Assessment (HTA) program.

Study design and setting: A retrospective cohort study of HTA monographs and corresponding journal publications including all clinical effectiveness randomized controlled trials published as HTA monographs between 1999 and 2005 by the NHS HTA program.

Results: There was a higher median P-value (P=0.33, interquartile range [IQR]: 0.02-0.54) among trials without a journal publication compared with those with a journal publication (P=0.14, IQR: 0.007-0.43), although the difference was not statistically significant (Mann-Whitney U test, z=-0.70; P=0.48). A higher proportion of statistically significant findings were reported in journal articles when compared with the outcomes reported in the HTA monographs. Trials published in general medical journals tended to have smaller P-values (median: 0.05, IQR: 0.001-0.22) than those published in more specialist journals (median: 0.33 IQR: 0.008-0.58), although this result was not significant (Mann-Whitney U test, z=-1.63; P=0.10).

Conclusions: Among journal-published trials, there were a greater proportion of statistically significant findings included in the journal reports compared with those in the HTA monographs.