Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2016 Nov 25:7:1832.
doi: 10.3389/fpsyg.2016.01832. eCollection 2016.

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

Jelte M Wicherts  1 , Coosje L S Veldkamp  1 , Hilde E M Augusteijn  1 , Marjan Bakker  1 , Robbie C M van Aert  1 , Marcel A L M van Assen  1
Affiliations
Review

Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking

Jelte M Wicherts et al. Front Psychol. .

Abstract

The designing, collecting, analyzing, and reporting of psychological studies entail many choices that are often arbitrary. The opportunistic use of these so-called researcher degrees of freedom aimed at obtaining statistically significant results is problematic because it enhances the chances of false positive results and may inflate effect size estimates. In this review article, we present an extensive list of 34 degrees of freedom that researchers have in formulating hypotheses, and in designing, running, analyzing, and reporting of psychological research. The list can be used in research methods education, and as a checklist to assess the quality of preregistrations and to determine the potential for bias due to (arbitrary) choices in unregistered studies.

Keywords: bias; experimental design; p-hacking; questionable research practices; research methods education; significance chasing; significance testing.

PubMed Disclaimer

References

    1. Asendorpf J. B., Conner M., Fruyt F. D., Houwer J. D., Denissen J. J. A., Fiedler K., et al. (2013). Recommendations for increasing replicability in psychology. Eur. J. Personal. 27 108–119. 10.1002/per.1919 - DOI
    1. Bakker M., Hartgerink C. H., Wicherts J. M., van der Maas H. L. (2016). Researchers’ Intuitions about power in psychological research. Psychol. Sci. 27 1069–1077. 10.1177/0956797616647519 - DOI - PMC - PubMed
    1. Bakker M., van Dijk A., Wicherts J. M. (2012). The rules of the game called psychological science. Perspect. Psychol. Sci. 7 543–554. 10.1177/1745691612459060 - DOI - PubMed
    1. Bakker M., Wicherts J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behav. Res. Methods 43 666–678. 10.3758/s13428-011-0089-5 - DOI - PMC - PubMed
    1. Bakker M., Wicherts J. M. (2014). Outlier removal, sum scores, and the inflation of the Type I error rate in independent samples t tests. The power of alternatives and recommendations. Psychol. Methods 19 409–427. 10.1037/met0000014 - DOI - PubMed

LinkOut - more resources