The replication crisis has led to positive structural, procedural, and community changes

Max Korbmacher^{1

2

3}, Flavio Azevedo^{4

5}, Charlotte R Pennington⁶, Helena Hartmann⁷, Madeleine Pownall⁸, Kathleen Schmidt⁹, Mahmoud Elsherif¹⁰, Nate Breznau¹¹, Olly Robertson¹², Tamara Kalandadze¹³, Shijun Yu¹⁰, Bradley J Baker¹⁴, Aoife O'Mahony¹⁵, Jørgen Ø-S Olsnes¹⁶, John J Shaw¹⁷, Biljana Gjoneska¹⁸, Yuki Yamada¹⁹, Jan P Röer²⁰, Jennifer Murphy²¹, Shilaan Alzahawi²², Sandra Grinschgl²³, Catia M Oliveira²⁴, Tobias Wingen²⁵, Siu Kit Yeung²⁶, Meng Liu²⁷, Laura M König²⁸, Nihan Albayrak-Aydemir^{29

30}, Oscar Lecuona^{31

32}, Leticia Micheli³³, Thomas Evans^{34

35}

Affiliations

¹ Department of Health and Functioning, Western Norway University of Applied Sciences, Bergen, Norway.
² NORMENT Centre for Psychosis Research, University of Oslo and Oslo University Hospital, Oslo, Norway.
³ Mohn Medical Imaging and Visualisation Center, Bergen, Norway.
⁴ Department of Psychology, University of Cambridge, Cambridge, UK. flavio.azevedo@rug.nl.
⁵ Department of Social Psychology, University of Groningen, Groningen, The Netherlands. flavio.azevedo@rug.nl.
⁶ School of Psychology, Aston University, Birmingham, UK.
⁷ Department of Neurology, University of Essen, Essen, Germany.
⁸ School of Psychology, University of Leeds, Leeds, UK.
⁹ Department of Psychology, Ashland University, Ashland, USA.
¹⁰ School of Psychology, University of Birmingham, Birmingham, UK.
¹¹ SOCIUM Research Center on Inequality and Social Policy, University of Bremen, Bremen, Germany.
¹² Department of Psychiatry, University of Oxford, Oxford, UK.
¹³ Department of Education, ICT and Learning, Ostfold University College, Halden, Norway.
¹⁴ Department of Sport and Recreation Management, Temple University, Philadelphia, USA.
¹⁵ School of Psychology, Cardiff University, Cardiff, UK.
¹⁶ Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway.
¹⁷ Division of Psychology, De Montfort University, Leicester, UK.
¹⁸ Macedonian Academy of Sciences and Arts, Skopje, North Macedonia.
¹⁹ Faculty of Arts and Science, Kyushu University, Fukuoka, Japan.
²⁰ Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Germany.
²¹ Department of Applied Science, Technological University Dublin, Dublin, Ireland.
²² Graduate School of Business, Stanford University, Standford, USA.
²³ Institute of Psychology, University of Graz, Graz, Austria.
²⁴ Department of Psychology, University of York, York, UK.
²⁵ Institute of General Practice and Family Medicine, University of Bonn, Bonn, Germany.
²⁶ Department of Psychology, Chinese University of Hong Kong, Hong Kong, China.
²⁷ Faculty of Education, University of Cambridge, Cambridge, UK.
²⁸ Faculty of Life Sciences: Food, Nutrition and Health, University of Bayreuth, Bayreuth, Germany.
²⁹ Open Psychology Research Centre, Open University, Milton Keynes, UK.
³⁰ Department of Psychological and Behavioural Science, London School of Economics and Political Science, London, UK.
³¹ Department of Psychology, Universidad Rey Juan Carlos, Madrid, Spain.
³² Faculty of Psychology, Universidad Autónoma de Madrid, Madrid, Spain.
³³ Institute of Psychology, Leiden University, Leiden, The Netherlands.
³⁴ School of Human Sciences, University of Greenwich, Greenwich, UK.
³⁵ Institute for Lifecourse Development, University of Greenwich, Greenwich, UK.

PMID: 39242883
PMCID: PMC11290608
DOI: 10.1038/s44271-023-00003-2

Review

The replication crisis has led to positive structural, procedural, and community changes

Max Korbmacher et al. Commun Psychol. 2023.

. 2023 Jul 25;1(1):3.

doi: 10.1038/s44271-023-00003-2.

Authors

Affiliations

¹ Department of Health and Functioning, Western Norway University of Applied Sciences, Bergen, Norway.
² NORMENT Centre for Psychosis Research, University of Oslo and Oslo University Hospital, Oslo, Norway.
³ Mohn Medical Imaging and Visualisation Center, Bergen, Norway.
⁴ Department of Psychology, University of Cambridge, Cambridge, UK. flavio.azevedo@rug.nl.
⁵ Department of Social Psychology, University of Groningen, Groningen, The Netherlands. flavio.azevedo@rug.nl.
⁶ School of Psychology, Aston University, Birmingham, UK.
⁷ Department of Neurology, University of Essen, Essen, Germany.
⁸ School of Psychology, University of Leeds, Leeds, UK.
⁹ Department of Psychology, Ashland University, Ashland, USA.
¹⁰ School of Psychology, University of Birmingham, Birmingham, UK.
¹¹ SOCIUM Research Center on Inequality and Social Policy, University of Bremen, Bremen, Germany.
¹² Department of Psychiatry, University of Oxford, Oxford, UK.
¹³ Department of Education, ICT and Learning, Ostfold University College, Halden, Norway.
¹⁴ Department of Sport and Recreation Management, Temple University, Philadelphia, USA.
¹⁵ School of Psychology, Cardiff University, Cardiff, UK.
¹⁶ Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway.
¹⁷ Division of Psychology, De Montfort University, Leicester, UK.
¹⁸ Macedonian Academy of Sciences and Arts, Skopje, North Macedonia.
¹⁹ Faculty of Arts and Science, Kyushu University, Fukuoka, Japan.
²⁰ Department of Psychology and Psychotherapy, Witten/Herdecke University, Witten, Germany.
²¹ Department of Applied Science, Technological University Dublin, Dublin, Ireland.
²² Graduate School of Business, Stanford University, Standford, USA.
²³ Institute of Psychology, University of Graz, Graz, Austria.
²⁴ Department of Psychology, University of York, York, UK.
²⁵ Institute of General Practice and Family Medicine, University of Bonn, Bonn, Germany.
²⁶ Department of Psychology, Chinese University of Hong Kong, Hong Kong, China.
²⁷ Faculty of Education, University of Cambridge, Cambridge, UK.
²⁸ Faculty of Life Sciences: Food, Nutrition and Health, University of Bayreuth, Bayreuth, Germany.
²⁹ Open Psychology Research Centre, Open University, Milton Keynes, UK.
³⁰ Department of Psychological and Behavioural Science, London School of Economics and Political Science, London, UK.
³¹ Department of Psychology, Universidad Rey Juan Carlos, Madrid, Spain.
³² Faculty of Psychology, Universidad Autónoma de Madrid, Madrid, Spain.
³³ Institute of Psychology, Leiden University, Leiden, The Netherlands.
³⁴ School of Human Sciences, University of Greenwich, Greenwich, UK.
³⁵ Institute for Lifecourse Development, University of Greenwich, Greenwich, UK.

PMID: 39242883
PMCID: PMC11290608
DOI: 10.1038/s44271-023-00003-2

Abstract

The emergence of large-scale replication projects yielding successful rates substantially lower than expected caused the behavioural, cognitive, and social sciences to experience a so-called 'replication crisis'. In this Perspective, we reframe this 'crisis' through the lens of a credibility revolution, focusing on positive structural, procedural and community-driven changes. Second, we outline a path to expand ongoing advances and improvements. The credibility revolution has been an impetus to several substantive changes which will have a positive, long-term impact on our research environment.

PubMed Disclaimer

Conflict of interest statement

This work is as an initiative from The Framework for Open and Reproducible Research Training (FORRT; https://forrt.org). All authors are advocates of open scholarship with different open science organisational affiliations, including FORRT: M.K., F.A., H.H., M.M.E., A.O.M., B.S., T.K., C.R.P., J.J.S., Y.Y., J.P.R., J.M., S.A., B.J.B., S.G., C.M.O., T.W., S.K.Y., M.L., L.M.K., N.A.A., O.L., T.E., O.R., L.M., K.S. and S.Y. ReproducibiliTEA journal clubs: Y.Y., J.S., C.R.P., M.M.E. and O.L. NASA TOPs: F.A. and S.A. Student Initiative for Open Science: M.K. Open Applied Linguistics: M.L. Psychological Science Accelerator: M.K., F.A., T.K., C.R.P., M.P., K.S., M.E., N.B., B.B., A.O.M., B.G., Y.Y., J.R.P., S.A., S.K.Y., M.L., N.A.A., T.E. and M.M.E. ManyMany: B.G. and M.M.E. ManyBabies: M.M.E. The United Kingdom Reproducibility Network: F.A., C.R.P., T.E., O.R., C.R.P. and A.O.M. Reproducible Research Oxford: O.R. The Sports Science Replication Centre: J.M. Norway’s Reproducibility Network: T.K. and M.K. Interessengruppe Offene und Reproduzierbare Forschung: H.H. Arbeitsgruppe Open Science der Fachgruppe Gesundheitspsychologie in der Deutschen Gesellschaft für Psychologie: L.M.K. Reproducible Interpretable Open Transparent (RIOT) Science Club: T.K. Center for Open Science: S.A. Open Sciency: F.A. and S.A. Society for the Improvement of Psychological Science: F.A. and S.A. Stanford Center for Open and Reproducible Science: S.A.

Figures

**Fig. 1. Modes of change towards scientific credibility.**
This figure presents an overview of the three modes of change proposed in this article: structural change is often evoked at the institutional level and expressed by new norms and rules; procedural change refers to behaviours and sets of commonly used practices in the research process; community change encompasses how work and collaboration within the scientific community evolves.

See this image and copyright information in PMC

References

1. Bem, D. Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. J. Pers. Soc. Psychol.100, 407 (2011). 10.1037/a0021524 - DOI - PubMed
1. Crocker, J. The road to fraud starts with a single step. Nature479, 151–151 (2011). 10.1038/479151a - DOI - PubMed
1. Wagenmakers, E.-J., Wetzels, R., Borsboom, D. & Van Der Maas, H. L. Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). JPSP. 100, 426–432 (2011). - PubMed
1. Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav.1, 1–9 (2017). 10.1038/s41562-016-0021 - DOI - PMC - PubMed
1. Open Science Collaboration. Estimating the reproducibility of psychological science. Science349, aac4716 (2015). This study was one of the first large-scale replication projects showing lower replication rates and smaller effect sizes among “successful” replicated findings. 10.1126/science.aac4716 - DOI - PubMed

Publication types

Actions

LinkOut - more resources

Full Text Sources
- Bern Open Repository and Information System
- PubMed Central
Medical
- The YODA Project

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

The replication crisis has led to positive structural, procedural, and community changes

Affiliations

The replication crisis has led to positive structural, procedural, and community changes

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

LinkOut - more resources

Full Text Sources

Medical