Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention

Abstract

Recent systematic reviews and empirical evaluations of the cognitive sciences literature suggest that publication and other reporting biases are prevalent across diverse domains of cognitive science. In this review, we summarize the various forms of publication and reporting biases and other questionable research practices, and overview the available methods for probing into their existence. We discuss the available empirical evidence for the presence of such biases across the neuroimaging, animal, other preclinical, psychological, clinical trials, and genetics literature in the cognitive sciences. We also highlight emerging solutions (from study design to data analyses and reporting) to prevent bias and improve the fidelity in the field of cognitive science research.

Keywords: bias; cognitive sciences; neuroscience; publication bias; reporting bias.

Figure 1. Brain Imaging Studies over the Last 50 Years

Number of brain imaging studies published in PubMed (search term “neuroimaging”, updated up to 2012) over the past 50 years.

Figure 2. Common Practices and Possible Solutions

Common practices and possible solutions across the workflow for addressing publication biases. Red problems and green solutions are mostly controllable by researchers; Purple problems and blue solutions are mostly controllable by journal editors. Funding agencies may also be major players in shaping incentives and reward systems.