Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Feb 19;7(2):190806.
doi: 10.1098/rsos.190806. eCollection 2020 Feb.

An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)

Tom E Hardwicke  1   2 , Joshua D Wallach  3   4   5 , Mallory C Kidwell  6 , Theiss Bendixen  7 , Sophia Crüwell  1 , John P A Ioannidis  1   2   8
Affiliations

An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)

Tom E Hardwicke et al. R Soc Open Sci. .

Abstract

Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

Keywords: meta-research; open science; reproducibility; social sciences; transparency.

PubMed Disclaimer

Conflict of interest statement

In the past 36 months, J.D.W. received research support outside of the scope of this project through the Collaboration for Research Integrity and Transparency from the Laura and John Arnold Foundation, and through the Center for Excellence in Regulatory Science and Innovation (CERSI) at Yale University and the Mayo Clinic (U01FD005938). All other authors declare no competing interests.

Figures

Figure 1.
Figure 1.
Assessment of transparency and reproducibility-related research practices in the social sciences. Numbers inside bars indicate raw counts. ‘N’ refers to total articles assessed for a given indicator (which was contingent on study design, table 1).
See this image and copyright information in PMC

References

    1. Chalmers I, Glasziou P. 2009. Viewpoint avoidable waste in the production and reporting of research evidence. The Lancet 374, 86–89. (10.1016/s0140-6736(09)60329-9) - DOI - PubMed
    1. Ioannidis JPA. 2012. Why science is not necessarily self-correcting. Perspect. Psychol. Sci. 7, 645–654. (10.1177/1745691612464056) - DOI - PubMed
    1. Vazire S. 2017. Quality uncertainty erodes trust in science. Collabra: Psychol. 3, 1–5. (10.1525/collabra.74) - DOI
    1. Young NS, Ioannidis JPA, Al-Ubaydli O. 2008. Why current publication practices may distort science. PLoS Med. 5, e201 (10.1371/journal.pmed.0050201) - DOI - PMC - PubMed
    1. Klein O, et al. 2018. A practical guide for transparency in psychological science. Collabra: Psychol. 4, 20 (10.1525/collabra.158) - DOI

LinkOut - more resources