#EEGManyLabs: Investigating the replicability of influential EEG experiments

Yuri G Pavlov¹, Nika Adamian², Stefan Appelhoff³, Mahnaz Arvaneh⁴, Christopher S Y Benwell⁵, Christian Beste⁶, Amy R Bland⁷, Daniel E Bradford⁸, Florian Bublatzky⁹, Niko A Busch¹⁰, Peter E Clayson¹¹, Damian Cruse¹², Artur Czeszumski¹³, Anna Dreber¹⁴, Guillaume Dumas¹⁵, Benedikt Ehinger¹⁶, Giorgio Ganis¹⁷, Xun He¹⁸, José A Hinojosa¹⁹, Christoph Huber-Huber²⁰, Michael Inzlicht²¹, Bradley N Jack²², Magnus Johannesson²³, Rhiannon Jones²⁴, Evgenii Kalenkovich²⁵, Laura Kaltwasser²⁶, Hamid Karimi-Rouzbahani²⁷, Andreas Keil²⁸, Peter König²⁹, Layla Kouara³⁰, Louisa Kulke³¹, Cecile D Ladouceur³², Nicolas Langer³³, Heinrich R Liesefeld³⁴, David Luque³⁵, Annmarie MacNamara³⁶, Liad Mudrik³⁷, Muthuraman Muthuraman³⁸, Lauren B Neal³⁹, Gustav Nilsonne⁴⁰, Guiomar Niso⁴¹, Sebastian Ocklenburg⁴², Robert Oostenveld²⁰, Cyril R Pernet⁴³, Gilles Pourtois⁴⁴, Manuela Ruzzoli⁴⁵, Sarah M Sass⁴⁶, Alexandre Schaefer⁴⁷, Magdalena Senderecka⁴⁸, Joel S Snyder⁴⁹, Christian K Tamnes⁵⁰, Emmanuelle Tognoli⁵¹, Marieke K van Vugt⁵², Edelyn Verona¹¹, Robin Vloeberghs⁵³, Dominik Welke⁵⁴, Jan R Wessel⁵⁵, Ilya Zakharov⁵⁶, Faisal Mushtaq⁵⁷

Affiliations

¹ University of Tuebingen, Germany; Ural Federal University, Russia. Electronic address: pavlovug@gmail.com.
² University of Aberdeen, UK.
³ Max Planck Institute for Human Development, Berlin, Germany.
⁴ University of Sheffield, UK.
⁵ University of Dundee, UK.
⁶ TU Dresden, Germany.
⁷ Manchester Metropolitan University, UK.
⁸ University of Miami, USA.
⁹ Heidelberg University, Germany.
¹⁰ University of Münster, Germany.
¹¹ University of South Florida, USA.
¹² University of Birmingham, UK.
¹³ University Osnabrück, Germany.
¹⁴ Stockholm School of Economics, Sweden; University of Innsbruck, Austria.
¹⁵ Université de Montréal, Montréal, Quebec, Canada; CHU Sainte-Justine Research Center, Montréal, Quebec, Canada.
¹⁶ University of Stuttgart, Germany.
¹⁷ University of Plymouth, UK.
¹⁸ Bournemouth University, UK.
¹⁹ Universidad Complutense de Madrid, Spain; Universidad Nebrija, Spain.
²⁰ Radboud University, Nijmegen, Netherlands.
²¹ University of Toronto, Canada.
²² The Australian National University, Canberra, Australia.
²³ Stockholm School of Economics, Sweden.
²⁴ Department of Psychology, University of Winchester, UK.
²⁵ HSE University, Moscow, Russia.
²⁶ Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Germany.
²⁷ University of Cambridge, UK; Macquarie University, Sydney, Australia.
²⁸ University of Florida, USA.
²⁹ University Osnabrück, Germany; University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
³⁰ University of Leeds, UK.
³¹ Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany.
³² University of Pittsburgh, USA.
³³ University of Zurich, Switzerland; Neuroscience Center Zurich, Switzerland.
³⁴ University of Bremen, Germany; Ludwig-Maximilians-Universität München, Germany.
³⁵ Universidad Autónoma de Madrid, Spain; Universidad de Málaga, Spain.
³⁶ Texas A&M University, USA.
³⁷ School of Psychological Sciences & Sagol School of Neuroscience, Tel Aviv University, Israel.
³⁸ Johannes Gutenberg University, Mainz, Germany.
³⁹ University of Texas Permian Basin, USA.
⁴⁰ Karolinska Institutet, Sweden; Stockholm University, Sweden.
⁴¹ Indiana University, Bloomington, USA; Universidad Politecnica de Madrid and CIBER-BBN, Spain.
⁴² Institute of Cognitive Neuroscience, Ruhr University Bochum, Germany.
⁴³ University of Edinburgh, UK.
⁴⁴ CAPLAB - Ghent University, Belgium.
⁴⁵ University of Glasgow, Glasgow, UK.
⁴⁶ The University of Texas at Tyler, USA.
⁴⁷ Monash University (Malaysia Campus), Malaysia.
⁴⁸ Institute of Philosophy, Jagiellonian University, Krakow, Poland.
⁴⁹ Department of Psychology, University of Nevada, Las Vegas, USA.
⁵⁰ University of Oslo, Oslo, Norway.
⁵¹ Florida Atlantic University, USA.
⁵² University of Groningen, the Netherlands.
⁵³ KU Leuven, Belgium.
⁵⁴ Max-Planck-Institute for Empirical Aesthetics, Germany.
⁵⁵ University of Iowa Hospitals and Clinics, Iowa City, USA; University of Iowa, Iowa City, USA.
⁵⁶ Russian Academy of Education, Russia.
⁵⁷ University of Leeds, UK. Electronic address: f.mushtaq@leeds.ac.uk.

PMID: 33965167
DOI: 10.1016/j.cortex.2021.03.013

Free article

#EEGManyLabs: Investigating the replicability of influential EEG experiments

Yuri G Pavlov et al. Cortex. 2021 Nov.

Free article

. 2021 Nov:144:213-229.

doi: 10.1016/j.cortex.2021.03.013. Epub 2021 Apr 2.

Authors

Affiliations

¹ University of Tuebingen, Germany; Ural Federal University, Russia. Electronic address: pavlovug@gmail.com.
² University of Aberdeen, UK.
³ Max Planck Institute for Human Development, Berlin, Germany.
⁴ University of Sheffield, UK.
⁵ University of Dundee, UK.
⁶ TU Dresden, Germany.
⁷ Manchester Metropolitan University, UK.
⁸ University of Miami, USA.
⁹ Heidelberg University, Germany.
¹⁰ University of Münster, Germany.
¹¹ University of South Florida, USA.
¹² University of Birmingham, UK.
¹³ University Osnabrück, Germany.
¹⁴ Stockholm School of Economics, Sweden; University of Innsbruck, Austria.
¹⁵ Université de Montréal, Montréal, Quebec, Canada; CHU Sainte-Justine Research Center, Montréal, Quebec, Canada.
¹⁶ University of Stuttgart, Germany.
¹⁷ University of Plymouth, UK.
¹⁸ Bournemouth University, UK.
¹⁹ Universidad Complutense de Madrid, Spain; Universidad Nebrija, Spain.
²⁰ Radboud University, Nijmegen, Netherlands.
²¹ University of Toronto, Canada.
²² The Australian National University, Canberra, Australia.
²³ Stockholm School of Economics, Sweden.
²⁴ Department of Psychology, University of Winchester, UK.
²⁵ HSE University, Moscow, Russia.
²⁶ Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Germany.
²⁷ University of Cambridge, UK; Macquarie University, Sydney, Australia.
²⁸ University of Florida, USA.
²⁹ University Osnabrück, Germany; University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
³⁰ University of Leeds, UK.
³¹ Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany.
³² University of Pittsburgh, USA.
³³ University of Zurich, Switzerland; Neuroscience Center Zurich, Switzerland.
³⁴ University of Bremen, Germany; Ludwig-Maximilians-Universität München, Germany.
³⁵ Universidad Autónoma de Madrid, Spain; Universidad de Málaga, Spain.
³⁶ Texas A&M University, USA.
³⁷ School of Psychological Sciences & Sagol School of Neuroscience, Tel Aviv University, Israel.
³⁸ Johannes Gutenberg University, Mainz, Germany.
³⁹ University of Texas Permian Basin, USA.
⁴⁰ Karolinska Institutet, Sweden; Stockholm University, Sweden.
⁴¹ Indiana University, Bloomington, USA; Universidad Politecnica de Madrid and CIBER-BBN, Spain.
⁴² Institute of Cognitive Neuroscience, Ruhr University Bochum, Germany.
⁴³ University of Edinburgh, UK.
⁴⁴ CAPLAB - Ghent University, Belgium.
⁴⁵ University of Glasgow, Glasgow, UK.
⁴⁶ The University of Texas at Tyler, USA.
⁴⁷ Monash University (Malaysia Campus), Malaysia.
⁴⁸ Institute of Philosophy, Jagiellonian University, Krakow, Poland.
⁴⁹ Department of Psychology, University of Nevada, Las Vegas, USA.
⁵⁰ University of Oslo, Oslo, Norway.
⁵¹ Florida Atlantic University, USA.
⁵² University of Groningen, the Netherlands.
⁵³ KU Leuven, Belgium.
⁵⁴ Max-Planck-Institute for Empirical Aesthetics, Germany.
⁵⁵ University of Iowa Hospitals and Clinics, Iowa City, USA; University of Iowa, Iowa City, USA.
⁵⁶ Russian Academy of Education, Russia.
⁵⁷ University of Leeds, UK. Electronic address: f.mushtaq@leeds.ac.uk.

PMID: 33965167
DOI: 10.1016/j.cortex.2021.03.013

Abstract

There is growing awareness across the neuroscience community that the replicability of findings about the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardised analysis pipelines. Inspired by recent efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations.

Keywords: Cognitive neuroscience; EEG; ERP; Many labs; Open science; Replication.

PubMed Disclaimer

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

MC_UU_00005/17/MRC_/Medical Research Council/United Kingdom

LinkOut - more resources

Full Text Sources
- Elsevier Science
- Zurich Open Access Repository and Archive
Other Literature Sources
- scite Smart Citations

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

#EEGManyLabs: Investigating the replicability of influential EEG experiments

Affiliations

#EEGManyLabs: Investigating the replicability of influential EEG experiments

Authors

Affiliations

Abstract

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources