Building a transdisciplinary expert consensus on the cognitive drivers of performance under pressure: An international multi-panel Delphi study

Lucy Albertella¹, Rebecca Kirkham¹, Amy B Adler², John Crampton³, Sean P A Drummond¹, Gerard J Fogarty⁴, James J Gross⁵, Leonard Zaichkowsky⁶, Judith P Andersen⁷, Paul T Bartone⁸, Danny Boga⁹, Jeffrey W Bond³, Tad T Brunyé¹⁰, Mark J Campbell¹¹, Liliana G Ciobanu¹², Scott R Clark¹², Monique F Crane¹³, Arne Dietrich¹⁴, Tracy J Doty², James E Driskell¹⁵, Ivar Fahsing¹⁶, Stephen M Fiore¹⁷, Rhona Flin¹⁸, Joachim Funke¹⁹, Justine M Gatt^{20

21}, P A Hancock¹⁷, Craig Harper¹, Andrew Heathcote^{22

23}, Kristin J Heaton²⁴, Werner F Helsen²⁵, Erika K Hussey²⁶, Robin C Jackson²⁷, Sangeet Khemlani²⁸, William D S Killgore²⁹, Sabina Kleitman³⁰, Andrew M Lane³¹, Shayne Loft³², Clare MacMahon³³, Samuele M Marcora³⁴, Frank P McKenna³⁵, Carla Meijen³⁶, Vanessa Moulton³⁷, Gene M Moyle³⁸, Eugene Nalivaiko^{22

23}, Donna O'Connor³⁹, Dorothea O'Conor⁴⁰, Debra Patton⁴¹, Mark D Piccolo⁴², Coleman Ruiz⁴³, Linda Schücker⁴⁴, Ron A Smith⁴⁵, Sarah J R Smith⁴⁶, Chava Sobrino⁴⁷, Melba Stetz⁴⁸, Damien Stewart⁴⁹, Paul Taylor^{22

23}, Andrew J Tucker¹, Haike van Stralen⁵⁰, Joan N Vickers⁵¹, Troy A W Visser³², Rohan Walker^{22

23}, Mark W Wiggins¹³, Andrew Mark Williams⁵², Leonard Wong⁵³, Eugene Aidman^{22

54}, Murat Yücel¹

Affiliations

¹ Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Melbourne, VIC, Australia.
² Walter Reed Army Institute of Research, Silver Spring, MD, United States.
³ APS College of Sport and Exercise Psychologists, Melbourne, VIC, Australia.
⁴ School of Psychology and Wellbeing, University of Southern Queensland, Toowoomba, QLD, Australia.
⁵ Stanford University, Stanford, CA, United States.
⁶ Wheelock College of Education and Human Development, Boston University, Boston, MA, United States.
⁷ Department of Psychology, University of Toronto, Toronto, ON, Canada.
⁸ National Defense University, Washington, DC, United States.
⁹ Australian Army Psychology Corps, Canberra, ACT, Australia.
¹⁰ U.S. Army DEVCOM Analysis Center, Natick, MA, United States.
¹¹ Physical Education & Sport Sciences Department, University of Limerick, Limerick, Ireland.
¹² Adelaide Medical School, University of Adelaide, Adelaide, SA, Australia.
¹³ School of Psychological Sciences, Macquarie University, Sydney, NSW, Australia.
¹⁴ Department of Psychology, American University of Beirut, Beirut, Lebanon.
¹⁵ Florida Maxima Corporation, Orlando, FL, United States.
¹⁶ Norwegian Police University College, Oslo, Norway.
¹⁷ Department of Psychology, and Institute of Simulation and Training, University of Central Florida, Orlando, FL, United States.
¹⁸ Aberdeen Business School, Robert Gordon University, Aberdeen, United Kingdom.
¹⁹ Department of Psychology, Heidelberg University, Heidelberg, Germany.
²⁰ School of Psychology, University of New South Wales, Kensington, NSW, Australia.
²¹ Neuroscience Research Australia, Sydney, NSW, Australia.
²² The University of Newcastle, Callaghan, NSW, Australia.
²³ School of Biomedical Sciences and Pharmacy, The University of Newcastle, Callaghan, NSW, Australia.
²⁴ US Army Research Institute of Environmental Medicine (USARIEM), Natick, MA, United States.
²⁵ Department of Movement Sciences, KU Leuven, Leuven, Belgium.
²⁶ Defense Innovation Unit, Mountain View, CA, United States.
²⁷ School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom.
²⁸ United States Naval Research Laboratory, Washington, DC, United States.
²⁹ Department of Psychiatry, University of Arizona, Tucson, AZ, United States.
³⁰ School of Psychology, The University of Sydney, Darlington, NSW, Australia.
³¹ Sport, Physical Activity Research Centre (SPARC), School of Sport, University of Wolverhampton, Wolverhampton, United Kingdom.
³² School of Psychological Science, University of Western Australia, Perth, WA, Australia.
³³ School of Allied Health, Human Services, and Sport, La Trobe University, Melbourne, VIC, Australia.
³⁴ Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy.
³⁵ Department of Psychology, University of Reading, Reading, United Kingdom.
³⁶ Faculty of Sport, Allied Health and Performance Science, St Mary's University, Twickenham, United Kingdom.
³⁷ Mindflex Group Ltd, London, United Kingdom.
³⁸ Faculty of Creative Industries, Education and Social Justice, Queensland University of Technology, Brisbane, QLD, Australia.
³⁹ Sydney School of Education and Social Work, The University of Sydney, Darlington, NSW, Australia.
⁴⁰ Department of Defense, Canberra, ACT, Australia.
⁴¹ United States Department of Defense, Washington DC, United States.
⁴² Fire Rescue Victoria, Melbourne, VIC, Australia.
⁴³ Mission Critical Team Institute, Annapolis, MD, United States.
⁴⁴ Department of Sport Psychology, Institute of Sport and Exercise Sciences, University of Münster, Münster, Germany.
⁴⁵ Consultant, Sydney, NSW, Australia.
⁴⁶ Defense Science and Technology Laboratory, Salisbury, United Kingdom.
⁴⁷ NSW Institute of Sport and Diving, Sydney, NSW, Australia.
⁴⁸ Independent Practitioner, Grand Ledge, MI, United States.
⁴⁹ Room23 Psychology, Brisbane, QLD, Australia.
⁵⁰ Altrecht Institute for Mental Health Care, Altrecht, Netherlands.
⁵¹ Faculty of Kinesiology, University of Calgary, Calgary, AB, Canada.
⁵² Florida Institute for Human and Machine Cognition, Florida, IL, United States.
⁵³ United States Army War College, Carlisle, PA, United States.
⁵⁴ Decision Sciences Division, Defense Science and Technology Group, Adelaide, SA, Australia.

PMID: 36755983
PMCID: PMC9901503
DOI: 10.3389/fpsyg.2022.1017675

Building a transdisciplinary expert consensus on the cognitive drivers of performance under pressure: An international multi-panel Delphi study

Lucy Albertella et al. Front Psychol. 2023.

. 2023 Jan 18:13:1017675.

doi: 10.3389/fpsyg.2022.1017675. eCollection 2022.

Authors

Affiliations

¹ Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Melbourne, VIC, Australia.
² Walter Reed Army Institute of Research, Silver Spring, MD, United States.
³ APS College of Sport and Exercise Psychologists, Melbourne, VIC, Australia.
⁴ School of Psychology and Wellbeing, University of Southern Queensland, Toowoomba, QLD, Australia.
⁵ Stanford University, Stanford, CA, United States.
⁶ Wheelock College of Education and Human Development, Boston University, Boston, MA, United States.
⁷ Department of Psychology, University of Toronto, Toronto, ON, Canada.
⁸ National Defense University, Washington, DC, United States.
⁹ Australian Army Psychology Corps, Canberra, ACT, Australia.
¹⁰ U.S. Army DEVCOM Analysis Center, Natick, MA, United States.
¹¹ Physical Education & Sport Sciences Department, University of Limerick, Limerick, Ireland.
¹² Adelaide Medical School, University of Adelaide, Adelaide, SA, Australia.
¹³ School of Psychological Sciences, Macquarie University, Sydney, NSW, Australia.
¹⁴ Department of Psychology, American University of Beirut, Beirut, Lebanon.
¹⁵ Florida Maxima Corporation, Orlando, FL, United States.
¹⁶ Norwegian Police University College, Oslo, Norway.
¹⁷ Department of Psychology, and Institute of Simulation and Training, University of Central Florida, Orlando, FL, United States.
¹⁸ Aberdeen Business School, Robert Gordon University, Aberdeen, United Kingdom.
¹⁹ Department of Psychology, Heidelberg University, Heidelberg, Germany.
²⁰ School of Psychology, University of New South Wales, Kensington, NSW, Australia.
²¹ Neuroscience Research Australia, Sydney, NSW, Australia.
²² The University of Newcastle, Callaghan, NSW, Australia.
²³ School of Biomedical Sciences and Pharmacy, The University of Newcastle, Callaghan, NSW, Australia.
²⁴ US Army Research Institute of Environmental Medicine (USARIEM), Natick, MA, United States.
²⁵ Department of Movement Sciences, KU Leuven, Leuven, Belgium.
²⁶ Defense Innovation Unit, Mountain View, CA, United States.
²⁷ School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom.
²⁸ United States Naval Research Laboratory, Washington, DC, United States.
²⁹ Department of Psychiatry, University of Arizona, Tucson, AZ, United States.
³⁰ School of Psychology, The University of Sydney, Darlington, NSW, Australia.
³¹ Sport, Physical Activity Research Centre (SPARC), School of Sport, University of Wolverhampton, Wolverhampton, United Kingdom.
³² School of Psychological Science, University of Western Australia, Perth, WA, Australia.
³³ School of Allied Health, Human Services, and Sport, La Trobe University, Melbourne, VIC, Australia.
³⁴ Department of Biomedical and Neuromotor Sciences, University of Bologna, Bologna, Italy.
³⁵ Department of Psychology, University of Reading, Reading, United Kingdom.
³⁶ Faculty of Sport, Allied Health and Performance Science, St Mary's University, Twickenham, United Kingdom.
³⁷ Mindflex Group Ltd, London, United Kingdom.
³⁸ Faculty of Creative Industries, Education and Social Justice, Queensland University of Technology, Brisbane, QLD, Australia.
³⁹ Sydney School of Education and Social Work, The University of Sydney, Darlington, NSW, Australia.
⁴⁰ Department of Defense, Canberra, ACT, Australia.
⁴¹ United States Department of Defense, Washington DC, United States.
⁴² Fire Rescue Victoria, Melbourne, VIC, Australia.
⁴³ Mission Critical Team Institute, Annapolis, MD, United States.
⁴⁴ Department of Sport Psychology, Institute of Sport and Exercise Sciences, University of Münster, Münster, Germany.
⁴⁵ Consultant, Sydney, NSW, Australia.
⁴⁶ Defense Science and Technology Laboratory, Salisbury, United Kingdom.
⁴⁷ NSW Institute of Sport and Diving, Sydney, NSW, Australia.
⁴⁸ Independent Practitioner, Grand Ledge, MI, United States.
⁴⁹ Room23 Psychology, Brisbane, QLD, Australia.
⁵⁰ Altrecht Institute for Mental Health Care, Altrecht, Netherlands.
⁵¹ Faculty of Kinesiology, University of Calgary, Calgary, AB, Canada.
⁵² Florida Institute for Human and Machine Cognition, Florida, IL, United States.
⁵³ United States Army War College, Carlisle, PA, United States.
⁵⁴ Decision Sciences Division, Defense Science and Technology Group, Adelaide, SA, Australia.

PMID: 36755983
PMCID: PMC9901503
DOI: 10.3389/fpsyg.2022.1017675

Erratum in

Erratum: Building a transdisciplinary expert consensus on the cognitive drivers of performance under pressure: an international multi-panel Delphi study.
Frontiers Production Office. Frontiers Production Office. Front Psychol. 2023 Jun 20;14:1236133. doi: 10.3389/fpsyg.2023.1236133. eCollection 2023. Front Psychol. 2023. PMID: 37408971 Free PMC article.

Abstract

Introduction: The ability to perform optimally under pressure is critical across many occupations, including the military, first responders, and competitive sport. Despite recognition that such performance depends on a range of cognitive factors, how common these factors are across performance domains remains unclear. The current study sought to integrate existing knowledge in the performance field in the form of a transdisciplinary expert consensus on the cognitive mechanisms that underlie performance under pressure.

Methods: International experts were recruited from four performance domains [(i) Defense; (ii) Competitive Sport; (iii) Civilian High-stakes; and (iv) Performance Neuroscience]. Experts rated constructs from the Research Domain Criteria (RDoC) framework (and several expert-suggested constructs) across successive rounds, until all constructs reached consensus for inclusion or were eliminated. Finally, included constructs were ranked for their relative importance.

Results: Sixty-eight experts completed the first Delphi round, with 94% of experts retained by the end of the Delphi process. The following 10 constructs reached consensus across all four panels (in order of overall ranking): (1) Attention; (2) Cognitive Control-Performance Monitoring; (3) Arousal and Regulatory Systems-Arousal; (4) Cognitive Control-Goal Selection, Updating, Representation, and Maintenance; (5) Cognitive Control-Response Selection and Inhibition/Suppression; (6) Working memory-Flexible Updating; (7) Working memory-Active Maintenance; (8) Perception and Understanding of Self-Self-knowledge; (9) Working memory-Interference Control, and (10) Expert-suggested-Shifting.

Discussion: Our results identify a set of transdisciplinary neuroscience-informed constructs, validated through expert consensus. This expert consensus is critical to standardizing cognitive assessment and informing mechanism-targeted interventions in the broader field of human performance optimization.

Keywords: assessment; cognition; expert consensus; high performance; transdisciplinary.

Copyright © 2023 Albertella, Kirkham, Adler, Crampton, Drummond, Fogarty, Gross, Zaichkowsky, Andersen, Bartone, Boga, Bond, Brunyé, Campbell, Ciobanu, Clark, Crane, Dietrich, Doty, Driskell, Fahsing, Fiore, Flin, Funke, Gatt, Hancock, Harper, Heathcote, Heaton, Helsen, Hussey, Jackson, Khemlani, Killgore, Kleitman, Lane, Loft, MacMahon, Marcora, McKenna, Meijen, Moulton, Moyle, Nalivaiko, O’Connor, O’Conor, Patton, Piccolo, Ruiz, Schücker, Smith, Smith, Sobrino, Stetz, Stewart, Taylor, Tucker, van Stralen, Vickers, Visser, Walker, Wiggins, Williams, Wong, Aidman and Yücel.

PubMed Disclaimer

Conflict of interest statement

VM was employed by Mindflex Group Ltd. JG is a stockholder in MAP Biotech Pty Ltd. SC has received speakers fees Janssen-Cilag Australia, Lundbeck Otsuka Australia, Servier Australia; Investigator Initiated research funding Janssen-Cilag Australia; Lundbeck Otsuka Australia; Advisory Boards Lundbeck Otsuka Australia. AT has received research funding from BHP, Rio Tinto, and Shell. SD is a Member of the Board of Advisors Eisai Australia Pty Ltd. MY has received payments in relation to court-, expert witness-, and/or expert review-reports. JD was employed by Florida Maxima Corporation. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

**Figure 1**
A visual representation of pre-Delphi and Delphi processes.

**Figure 2**
Mean ranking of the ten transdisciplinary constructs. Rank order is displayed within the corresponding marker. Error bars represent 95% Confidence Intervals. N.B. * denotes Cognitive Control subconstructs. ^ denotes Working Memory subconstructs. + denotes that Shifting was an expert-suggested construct (considered to belong in the Cognitive Systems Domain). ‘Goal Selection’ = Goal Selection; Updating; Representation; Maintenance. ‘Response Selection’ = Response Selection; Inhibition/Suppression.

See this image and copyright information in PMC

References

1. Aidman E. (2020). Cognitive fitness framework: toward assessing, training and augmenting individual-difference factors underpinning high-performance cognition. Front. Hum. Neurosci. 13:466. doi: 10.3389/fnhum.2019.00466, PMID: - DOI - PMC - PubMed
1. Baker J., Lovell K., Harris N. (2006). How expert are the experts? An exploration of the concept of ‘expert’within Delphi panel techniques. Nurs. Res. 14, 59–70. doi: 10.7748/nr2006.10.14.1.59.c6010, PMID: - DOI - PubMed
1. Barrett L. F., Gross J., Christensen T. C., Benvenuto M. (2001). Knowing what you're feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation. Cognit. Emot. 15, 713–724. doi: 10.1080/02699930143000239 - DOI
1. Bartone P. T., Adler A. B., Vaitkus M. A. (1998). Dimensions of psychological stress in peacekeeping operations. Mil. Med. 163, 587–593. doi: 10.1093/milmed/163.9.587, PMID: - DOI - PubMed
1. Biggs A. T., Pettijohn K. A. (2022). The role of inhibitory control in shoot/don’t-shoot decisions. Q. J. Exp. Psychol. 75, 536–549. doi: 10.1177/17470218211041923, PMID: - DOI - PubMed

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Building a transdisciplinary expert consensus on the cognitive drivers of performance under pressure: An international multi-panel Delphi study

Affiliations

Building a transdisciplinary expert consensus on the cognitive drivers of performance under pressure: An international multi-panel Delphi study

Authors

Affiliations

Erratum in

Abstract

Conflict of interest statement

Figures

References

LinkOut - more resources

Full Text Sources