Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee

Abstract

Rationale, aims and objectives: Within-study selective reporting is widely believed to exist, although to date there have been no empirical studies to assess the extent of the problem in clinical research. The present study aimed to examine this process.

Methods: We undertook a pilot study, involving a single local research ethics committee (LREC), in which we compared the outcomes, analysis and sample size proposed in the original approved study protocol with the results presented in the subsequent study report.

Results: We received 41 (73%) replies from lead researchers of 56 projects, which were a complete cohort of clinical research applications approved in a particular time period by the LREC. Fifteen of these projects, which were completed and published at the time of our study, were further investigated. Only six (40%) stated which outcome variables were of primary interest and four (67%) of these showed consistency in the reports. Eight (53%) of the 15 studies mentioned an analysis plan. However, seven (88%) of these eight studies did not follow their prescribed analysis plan: the analysis of outcome variables or associations between certain variables were found to be missing from the report.

Conclusions: Our pilot study has shown that within-study selective reporting may be examined qualitatively by comparing the study report with the study protocol. Our results suggest that it might well be substantial; however, the bias can only be broadly identified as protocols are not sufficiently precise.