Panel stacking is a threat to consensus statement validity

Kasper P Kepp¹, Preben Aavitsland², Marcel Ballin³, Francois Balloux⁴, Stefan Baral⁵, Kevin Bardosh⁶, Howard Bauchner⁷, Eran Bendavid⁸, Raj Bhopal⁹, Daniel T Blumstein¹⁰, Paolo Boffetta¹¹, Florence Bourgeois¹², Adam Brufsky¹³, Peter J Collignon¹⁴, Sally Cripps¹⁵, Ioana A Cristea¹⁶, Nigel Curtis¹⁷, Benjamin Djulbegovic¹⁸, Oliver Faude¹⁹, Maria Elena Flacco²⁰, Gordon H Guyatt²¹, George Hajishengallis²², Lars G Hemkens²³, Tammy Hoffmann²⁴, Ari R Joffe²⁵, Terry P Klassen²⁶, Despina Koletsi²⁷, Dimitrios P Kontoyiannis²⁸, Ellen Kuhl²⁹, Carlo La Vecchia³⁰, Tea Lallukka³¹, John Lambris³², Michael Levitt³³, Spyros Makridakis³⁴, Helena C Maltezou³⁵, Lamberto Manzoli³⁶, Ana Marusic³⁷, Clio Mavragani³⁸, David Moher³⁹, Ben W Mol⁴⁰, Taulant Muka⁴¹, Florian Naudet⁴², Paul W Noble⁴³, Anna Nordström⁴⁴, Peter Nordström⁴⁵, Nikolaos Pandis⁴⁶, Stefania Papatheodorou⁴⁷, Chirag J Patel⁴⁸, Irene Petersen⁴⁹, Stefan Pilz⁵⁰, Nikolaus Plesnila⁵¹, Anne-Louise Ponsonby⁵², Manuel A Rivas⁵³, Andrea Saltelli⁵⁴, Manuel Schabus⁵⁵, Michaéla C Schippers⁵⁶, Holger Schünemann⁵⁷, Marco Solmi⁵⁸, Andreas Stang⁵⁹, Hendrik Streeck⁶⁰, Joachim P Sturmberg⁶¹, Lehana Thabane⁵⁷, Brett D Thombs⁶², Athanasios Tsakris⁶³, Simon N Wood⁶⁴, John P A Ioannidis⁶⁵

Affiliations

¹ Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA.
² Pandemic Centre, University of Bergen, Bergen, Norway; Norwegian Institute of Public Health, Oslo, Norway.
³ Centre for Epidemiology and Community Medicine, Region Stockholm, Stockholm, Sweden; Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden.
⁴ UCL Genetics Institute, University College London, London, UK.
⁵ Department of Epidemiology, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of International Health, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of Health, Policy, and Management, Johns Hopkins School of Public Health, Baltimore, MD, USA.
⁶ School of Public Health, University of Washington, Seattle, WA, USA; Edinburgh Medical School, University of Edinburgh, Edinburgh, UK.
⁷ Department of Pediatrics, Boston University School of Medicine, Boston, MA, USA.
⁸ Department of Medicine (Primary Care and Population Health), Stanford University School of Medicine, Stanford, CA, USA; Department of Health Policy, Stanford University School of Medicine, Stanford, CA, USA; Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA, USA.
⁹ Usher Institute, University of Edinburgh, Edinburgh, UK.
¹⁰ Department of Ecology & Evolutionary Biology, Institute of the Environment & Sustainability, University of California Los Angeles, Los Angeles, CA, USA.
¹¹ Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy; Stony Brook Cancer Center, Stony Brook University, Stony Brook, NY, USA.
¹² Department of Pediatrics, Harvard Medical School, Boston, MA, USA.
¹³ Division of Hematology-Oncology, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA.
¹⁴ Department of Infectious Diseases and Microbiology, Canberra Hospital, Garran, Australian Capital Territory, Australia; Department of Infectious Disease, Medical School, Australian National University, Acton, Australian Capital Territory, Australia.
¹⁵ Human Technology Institute, University of Technology Sydney, Sydney, Australia.
¹⁶ Department of General Psychology, University of Padova, Padova, Italy.
¹⁷ Department of Paediatrics, The University of Melbourne, Parkville, Australia; Infectious Diseases Research Group, Murdoch Children's Research Institute, Parkville, Australia; Infectious Diseases Unit, The Royal Children's Hospital Melbourne, Parkville, Australia.
¹⁸ Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA.
¹⁹ Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland.
²⁰ Department of Environmental and Prevention Sciences, University of Ferrara, Ferrara, Italy.
²¹ Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Faculty of Health Sciences, Department of Medicine, McMaster University, Hamilton, Ontario, Canada.
²² Department of Basic and Translational Sciences, Penn Dental Medicine, University of Pennsylvania, Philadelphia, PA, USA.
²³ Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland.
²⁴ Faculty of Health Sciences and Medicine, Institute for Evidence-Based Healthcare, Bond University, Robina, Queensland, Australia.
²⁵ Department of Pediatrics and John Dossetor Health Ethics Center, University of Alberta, Edmonton, Alberta, Canada.
²⁶ Children's Hospital Research Institute of Manitoba, Department of Pediatrics and Child Health, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada.
²⁷ Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland.
²⁸ Division of Internal Medicine, Department of Infectious Diseases, Infection Control, and Employee Health, University of Texas MD Anderson Cancer Center, Houston, TX, USA.
²⁹ Departments of Mechanical Engineering and of Bioengineering, Stanford University, Stanford, CA, USA.
³⁰ Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milano, Italy.
³¹ Department of Public Health, University of Helsinki, Helsinki, Finland.
³² Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
³³ Department of Structural Biology, Stanford University School of Medicine, Stanford, CA, USA.
³⁴ Institute For the Future (IFF), University of Nicosia, Nicosia, Cyprus.
³⁵ Directorate of Research, Studies and Documentation, National Public Health Organization, Athens, Greece.
³⁶ Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy.
³⁷ Department of Research in Biomedicine and Health and Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia.
³⁸ Department of Physiology, Medical School, National and Kapodistrian University of Athens, Athens, Greece.
³⁹ Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁴⁰ Department of Obstetrics and Gynaecology, Monash University, Clayton, Australia.
⁴¹ Epistudia, Bern, Switzerland.
⁴² Research Institute for Environmental and Occupational Health (IRSET, UMR_S INSERM 1085), University of Rennes, Rennes, France; Institut Universitaire de France, Paris, France; Clinical Investigation Center (INSERM CIC 1414) and Adult Psychiatry Department, Rennes University Hospital, Rennes, France.
⁴³ Department of Medicine, Women's Guild Lung Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
⁴⁴ Department of Public Health and Clinical Medicine, Umeå University, Umeå, Sweden; School of Sport Sciences, UiT the Arctic University of Norway, Tromsø, Norway; Department of Health Sciences, The Swedish Winter Sport Research Centre, Mid Sweden University, Östersund, Sweden.
⁴⁵ Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden.
⁴⁶ Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland.
⁴⁷ Department of Biostatistics and Epidemiology, Rutgers School of Public Health, Piscataway, NJ, USA; Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
⁴⁸ Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
⁴⁹ Department of Primary Care and Population Health, University College London, London, UK.
⁵⁰ Division Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria.
⁵¹ Institute for Stroke and Dementia Research (ISD), Ludwig-Maximilians-University Munich, Munich, Germany; Munich Cluster for Systems Neurology (Synergy), Munich, Germany.
⁵² The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia; Murdoch Children's Research Institute, Royal Children's Hospital, Melbourne, Australia; Centre of Epidemiology and Biostatistics, School of Population and Global Health, University of Melbourne, Melbourne, Australia.
⁵³ Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA.
⁵⁴ UPF Barcelona School of Management, Barcelona, Spain; Centre for the Study of the Sciences and the Humanities, University of Bergen, Bergen, Norway.
⁵⁵ Department of Psychology, University of Salzburg, Salzburg, Austria.
⁵⁶ Department of Organisation and Personnel Management, Erasmus University Rotterdam, Rotterdam, Netherlands.
⁵⁷ Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
⁵⁸ Department of Psychiatry, University of Ottawa, Ottawa, Ontario, Canada; Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany.
⁵⁹ Institute of Medical Informatics, Biometry and Epidemiology, University Hospital Essen, Essen, Germany.
⁶⁰ Faculty of Medicine, Institute of Virology, University of Bonn, Bonn, Germany.
⁶¹ College of Health, Medicine and Wellbeing, University of Newcastle, Holgate, New South Wales, Australia; International Society for Systems and Complexity Sciences for Health, Waitsfield, Vermont, USA.
⁶² Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada; Department of Psychiatry, McGill University, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada; Department of Medicine, McGill University, Montreal, Quebec, Canada; Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada; Department of Psychology, McGill University, Montreal, Quebec, Canada.
⁶³ Department of Microbiology, Medical School, University of Athens, Athens, Greece.
⁶⁴ Chair of Computational Statistics, School of Mathematics, University of Edinburgh, Edinburgh, UK.
⁶⁵ Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Medicine (Stanford Prevention Research Center), Stanford University School of Medicine, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA. Electronic address: jioannid@stanford.edu.

PMID: 38897481
PMCID: PMC11913121
DOI: 10.1016/j.jclinepi.2024.111428

Panel stacking is a threat to consensus statement validity

Kasper P Kepp et al. J Clin Epidemiol. 2024 Sep.

. 2024 Sep:173:111428.

doi: 10.1016/j.jclinepi.2024.111428. Epub 2024 Jun 17.

Authors

Affiliations

¹ Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA.
² Pandemic Centre, University of Bergen, Bergen, Norway; Norwegian Institute of Public Health, Oslo, Norway.
³ Centre for Epidemiology and Community Medicine, Region Stockholm, Stockholm, Sweden; Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden.
⁴ UCL Genetics Institute, University College London, London, UK.
⁵ Department of Epidemiology, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of International Health, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of Health, Policy, and Management, Johns Hopkins School of Public Health, Baltimore, MD, USA.
⁶ School of Public Health, University of Washington, Seattle, WA, USA; Edinburgh Medical School, University of Edinburgh, Edinburgh, UK.
⁷ Department of Pediatrics, Boston University School of Medicine, Boston, MA, USA.
⁸ Department of Medicine (Primary Care and Population Health), Stanford University School of Medicine, Stanford, CA, USA; Department of Health Policy, Stanford University School of Medicine, Stanford, CA, USA; Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA, USA.
⁹ Usher Institute, University of Edinburgh, Edinburgh, UK.
¹⁰ Department of Ecology & Evolutionary Biology, Institute of the Environment & Sustainability, University of California Los Angeles, Los Angeles, CA, USA.
¹¹ Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy; Stony Brook Cancer Center, Stony Brook University, Stony Brook, NY, USA.
¹² Department of Pediatrics, Harvard Medical School, Boston, MA, USA.
¹³ Division of Hematology-Oncology, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA.
¹⁴ Department of Infectious Diseases and Microbiology, Canberra Hospital, Garran, Australian Capital Territory, Australia; Department of Infectious Disease, Medical School, Australian National University, Acton, Australian Capital Territory, Australia.
¹⁵ Human Technology Institute, University of Technology Sydney, Sydney, Australia.
¹⁶ Department of General Psychology, University of Padova, Padova, Italy.
¹⁷ Department of Paediatrics, The University of Melbourne, Parkville, Australia; Infectious Diseases Research Group, Murdoch Children's Research Institute, Parkville, Australia; Infectious Diseases Unit, The Royal Children's Hospital Melbourne, Parkville, Australia.
¹⁸ Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA.
¹⁹ Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland.
²⁰ Department of Environmental and Prevention Sciences, University of Ferrara, Ferrara, Italy.
²¹ Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Faculty of Health Sciences, Department of Medicine, McMaster University, Hamilton, Ontario, Canada.
²² Department of Basic and Translational Sciences, Penn Dental Medicine, University of Pennsylvania, Philadelphia, PA, USA.
²³ Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland.
²⁴ Faculty of Health Sciences and Medicine, Institute for Evidence-Based Healthcare, Bond University, Robina, Queensland, Australia.
²⁵ Department of Pediatrics and John Dossetor Health Ethics Center, University of Alberta, Edmonton, Alberta, Canada.
²⁶ Children's Hospital Research Institute of Manitoba, Department of Pediatrics and Child Health, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada.
²⁷ Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland.
²⁸ Division of Internal Medicine, Department of Infectious Diseases, Infection Control, and Employee Health, University of Texas MD Anderson Cancer Center, Houston, TX, USA.
²⁹ Departments of Mechanical Engineering and of Bioengineering, Stanford University, Stanford, CA, USA.
³⁰ Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milano, Italy.
³¹ Department of Public Health, University of Helsinki, Helsinki, Finland.
³² Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
³³ Department of Structural Biology, Stanford University School of Medicine, Stanford, CA, USA.
³⁴ Institute For the Future (IFF), University of Nicosia, Nicosia, Cyprus.
³⁵ Directorate of Research, Studies and Documentation, National Public Health Organization, Athens, Greece.
³⁶ Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy.
³⁷ Department of Research in Biomedicine and Health and Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia.
³⁸ Department of Physiology, Medical School, National and Kapodistrian University of Athens, Athens, Greece.
³⁹ Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁴⁰ Department of Obstetrics and Gynaecology, Monash University, Clayton, Australia.
⁴¹ Epistudia, Bern, Switzerland.
⁴² Research Institute for Environmental and Occupational Health (IRSET, UMR_S INSERM 1085), University of Rennes, Rennes, France; Institut Universitaire de France, Paris, France; Clinical Investigation Center (INSERM CIC 1414) and Adult Psychiatry Department, Rennes University Hospital, Rennes, France.
⁴³ Department of Medicine, Women's Guild Lung Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
⁴⁴ Department of Public Health and Clinical Medicine, Umeå University, Umeå, Sweden; School of Sport Sciences, UiT the Arctic University of Norway, Tromsø, Norway; Department of Health Sciences, The Swedish Winter Sport Research Centre, Mid Sweden University, Östersund, Sweden.
⁴⁵ Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden.
⁴⁶ Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland.
⁴⁷ Department of Biostatistics and Epidemiology, Rutgers School of Public Health, Piscataway, NJ, USA; Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
⁴⁸ Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
⁴⁹ Department of Primary Care and Population Health, University College London, London, UK.
⁵⁰ Division Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria.
⁵¹ Institute for Stroke and Dementia Research (ISD), Ludwig-Maximilians-University Munich, Munich, Germany; Munich Cluster for Systems Neurology (Synergy), Munich, Germany.
⁵² The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia; Murdoch Children's Research Institute, Royal Children's Hospital, Melbourne, Australia; Centre of Epidemiology and Biostatistics, School of Population and Global Health, University of Melbourne, Melbourne, Australia.
⁵³ Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA.
⁵⁴ UPF Barcelona School of Management, Barcelona, Spain; Centre for the Study of the Sciences and the Humanities, University of Bergen, Bergen, Norway.
⁵⁵ Department of Psychology, University of Salzburg, Salzburg, Austria.
⁵⁶ Department of Organisation and Personnel Management, Erasmus University Rotterdam, Rotterdam, Netherlands.
⁵⁷ Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
⁵⁸ Department of Psychiatry, University of Ottawa, Ottawa, Ontario, Canada; Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany.
⁵⁹ Institute of Medical Informatics, Biometry and Epidemiology, University Hospital Essen, Essen, Germany.
⁶⁰ Faculty of Medicine, Institute of Virology, University of Bonn, Bonn, Germany.
⁶¹ College of Health, Medicine and Wellbeing, University of Newcastle, Holgate, New South Wales, Australia; International Society for Systems and Complexity Sciences for Health, Waitsfield, Vermont, USA.
⁶² Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada; Department of Psychiatry, McGill University, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada; Department of Medicine, McGill University, Montreal, Quebec, Canada; Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada; Department of Psychology, McGill University, Montreal, Quebec, Canada.
⁶³ Department of Microbiology, Medical School, University of Athens, Athens, Greece.
⁶⁴ Chair of Computational Statistics, School of Mathematics, University of Edinburgh, Edinburgh, UK.
⁶⁵ Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Medicine (Stanford Prevention Research Center), Stanford University School of Medicine, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA. Electronic address: jioannid@stanford.edu.

PMID: 38897481
PMCID: PMC11913121
DOI: 10.1016/j.jclinepi.2024.111428

Abstract

Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members toward one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but nonfinancial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analyzing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated toward COVID-19 elimination (Zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases toward advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.

Keywords: Competing interests; Consensus statements; Evidence based medicine; Guidelines; Panel bias; Transparency.

PubMed Disclaimer

Conflict of interest statement

Declaration of competing interest K.P.K. declares no conflicts of interest. P.A. has as part of his work at the Norwegian Institute of Public Health (a governmental agency) provided advice to the Governments of Norway and Denmark on handling the COVID-19 pandemic. He was a member of the WHO Review Committee on the Functioning of the International Health Regulations (2005) during the COVID-19 Response, and he chaired the WHO Review Committee regarding Standing Recommendations for COVID. M.B. declares no conflicts of interest. F.B. is the leader of 2 work packages of the END-VOC project funded by the European Union’s Horizon Europe program under the grant agreement No. 101046314. He has published multiple academic and public outreach articles on COVID-19. He has given unpaid scientific advice on pandemic mitigation and vaccination policy to the Governments of Austria, France, and the United Kingdom. S.B. has published on COVID-19, did COVID-19–related clinical work, and was a co-author on a Royal Society report on homelessness during COVID-19 which included systematic reviews which were later published (https://rsc-src.ca/en/themes/homelessness). K.B. is Director of Collateral Global, a UK-based research and education charity that is focused on understanding the impact of COVID policies around the world. H.B. declares no conflicts of interest. E.B. has written COVID-19 research and opinion pieces (eg, https://www.wsj.com/articles/is-the-coronavirus-as-deadly-as-they-say-11585088464) on COVID-19, some questioning consensus on disease control interventions. R.B. has served on COVID-19–related committees of government and NGOs, including the Expert Reference Group on COVID-19 and Ethnicity, the Scottish Government, 2020-2022, and was a signature on several letters, including one to the Prime Minister (29/5 2020): “Dominic Cumming’s actions damage public trust” (https://www.theguardian.com/commentisfree/2020/may/30/dominic-cummings-actions-damage-public-health). D.T.B. is a member of the board of the International Panel on Behavior Change, an international group that seeks to integrate behavioral knowledge to improve global sustainability. P.B. is a co-PI of a grant on COVID-19 epidemic awarded by the European Commission to the University of Bologna. F.B. declares no conflicts of interest. A.B. receives consulting fees from AstraZeneca, Pfizer, Novartis, Lilly, Genentech/Roche, SeaGen, Daiichi Sankyo, Merck, Agendia, Sanofi, Puma, Myriad, and Gilead (unrelated to the current work). P.J.C. was member of Infection control expert group (ICEG) that provided advice on Infection control and prevention issues to Australian Government bodies, Federal Health department and Chief Health Officers during the COVID-19 pandemic from 2020 to 2022, expert witness for the Australian Government in dispute with Western Australia on closure of State borders, member of the review team assessing hotel quarantine procedures by states and territories for the Australian government, and member of the team advising the Federal health department in 2022 on the future potential use and volumes of antiviral and vaccines use for COVID-19. S.C. declares no conflicts of interest. I.A.C. declares no conflicts of interest. N.C. declares no conflicts of interest. B.D. declares no conflicts of interest. O.F. declares no conflicts of interest. M.E.F. declares no conflicts of interest. G.H.G. declares that he is on the editorial board of Journal of Clinical Epidemiology. G.H. declares no conflicts of interest. L.G.H. received funding for the COVID-evidence project (www.covid-evidence.org) by the Swiss National Science Foundation (project 31CA30_196190), unrelated to this work, and travel support from the WHO, unrelated to this work. L.G.H.’s institution (RC2NB) was contracted by WHO for the development of study protocol templates to evaluate Public Health and Social Measures, unrelated to this work. L.G.H.’s institution (RC2NB) is supported by Foundation Clinical Neuroimmunology and Neuroscience Basel, unrelated to this work. RC2NB has a contract with Roche for a steering committee participation of L.G.H., unrelated to this work. These funders had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript. He is also a member of the Network for Evidence-based Medicine (EbM-Netzwerk). T.H. declares no conflicts of interest. A.R.J. served as an unpaid advisor to the Public Health Emergencies Governance Review Panel in the province of Alberta, Canada, whose “report reviews legislation that guided Alberta’s response to COVID-19 and recommends changes to improve the handling of future public health emergencies for Albertans” (the final report was published in November 2023). He signed the Great Barrington Declaration. T.P.K. declares no conflicts of interest. D.K. declares no conflicts of interest. D.P.K. declares no conflicts of interest. E.K. declares no conflicts of interest. C.L.V. was a member of the Vaxzevria advisory board. T.L. has provided an invited report for the Committee for the Future (an established, standing committee in the Parliament of Finland) in late 2022, where she was asked to describe the future of public health. Her report covered, for example, inequalities in health, population aging, oral health, and mental health. J.L. declares no conflicts of interest. M.L. has signed the Great Barrington Declaration. S.M. declares no conflicts of interest. H.C.M. declares no conflicts of interest. L.M. declares no conflicts of interest. A.M. declares no conflicts of interest. C.M. declares no conflicts of interest. D.M. declares that he is on the editorial board of Journal of Clinical Epidemiology. B.M. is supported by an NHMRC Investigator grant (GNT1176437), and he reports consultancy, travel support, and research funding from Merck and consultancy for Organon and Norgine, and holding stock from ObsEva. T.M. is co-founder and CEO at Epistudia GmbH and acts as advisor for the Academic Parity Movement, a nonprofit organization uprooting academic bullying, discrimination, and violence. F.N. received funding from the French National Research Agency (ANR-17-CE36-0010), the French ministry of health, and the French ministry of research. He is a work package leader in the OSIRIS project (Open Science to Increase Reproducibility in Science). The OSIRIS project has received funding from the European Union’s Horizon Europe research and innovation program under the grant agreement No. 101094725. He is a work package leader for the doctoral network MSCA-DN SHARE-CTD (HORIZON-MSCA-2022-DN-01 101120360), funded by the EU, unpaid member of the French Society of clinical pharmacology and therapeutics, and unpaid member of the European society of clinical pharmacology and therapeutics. P.W.N. declares no conflicts of interest. A.N. declares no conflicts of interest. P.N. declares no conflicts of interest. N.P. declares no conflicts of interest. S.P. declares no conflicts of interest. C.J.P. declares no conflicts of interest. I.P. declares no conflicts of interest. S.P. has signed the Great Barrington Declaration. N.P. declares no conflicts of interest. A.L.P. owns stock in Dysrupt Labs, a subsidiary of slowVoice Pty Ltd. Dysrupt Labs supplied the Almanis prediction market database for an article recently published in eBioMedicine in which Professor Ponsonby was the corresponding author, entitled “Machine learning augmentation reduces prediction error in collective forecasting: development and validation across prediction markets with application to COVID events.” M.A.R. is a Co-Founder of Broadwing Bio and consults for insitro, Mubadala Ventures, and Curie.Bio. A.S. declares no conflicts of interest. M.S. declares no conflicts of interest. M.S. is founder of Ikigaitv.nl: positive psychology interventions for the general public, to enhance mental wellbeing, and co-founder of Great Citizens Movement (greatcitizensmovement.org). She has signed the Great Barrington Declaration. She also served as expert witness of extraparliamentary inquiry regarding the COVID-19 crisis handling in the Netherlands (2020); she owns the podcast followthescience.nl and does contract research on improving study success: https://www.erim.eur.nl/erasmus-centre-for-study-and-career-success/. H.S. is the lead of the steering committee for the International Guideline Training and Certification Program INGUIDE which is an ISO-certified program focusing on bringing professionalism to the development of health recommendations by educating and certifying those participating in the development of health recommendations; the program charges course fees, and time spent on teaching in INGUIDE may lead to reimbursement to HS in the future (until now no payments have been made). He is also co-chair of the GRADE Working Group and chair of the board of trustees of the Guidelines International Network, both are having transparency and trustworthy guideline development methods as a core value. He is PI on a research contract from the Public Health Agency of Canada that focuses on developing and implementing post–COVID-19 condition guidelines. He has been a PI on several grants focusing on guideline methods and knowledge mobilization related to COVID-19 guidelines. He has worked with numerous global and other organizations on methods and development of guidelines. M.S. has received honoraria/has been a consultant for AbbVie, Angelini, Lundbeck, and Otsuka, unrelated to this work. A.S. received COVID-19 research grants from the German Science Foundation (DFC) without any conflict of interest. The granting agency DFG is not affected at all by this manuscript. H.S. has been on the COVID-19 expert group pf the government of North Rhine-Westfalia, Germany 2020-2021, the COVID-19 expert group of the government of Germany 2021-2023, the expert group for evaluating infectious disease laws in Germany, since 2023 on the Enquete committee for the evaluation of crisis management of the government of North Rhine-Westfalia, and since 2019, head of the board of trustees of the German AIDS foundation. J.P.S. declares no conflicts of interest. L.T. has worked as a paid consultant by the companies Bausch Health, GSK, Teva Pharmaceuticals, and Theralase Inc and also works as Vice-President Research for St Joseph’s Healthcare Hamilton and he serves on several committees or boards of health-related organizations that include Ontario Hospital Association, HealthcareCan, the SPOR (Strategy for Patient-Oriented Research) Refresh Steering Committee of the Canadian Institutes of Health Research, the Canadian Medical Hall of Fame, Canadian Academy of Health Sciences, Society for Clinical Trials, Baycrest Academy, and the Patient-Centered Outcomes Research Institute Methodology Committee. B.D.T. is supported by a Canada Research Chair and was PI on several grants from the Canadian Institutes of Health Research to study mental health in COVID-19, and consulted with the Public Health Agency of Canada on this topic. A.T. was a member of the National Greek Committee for the Protection of Public Health against COVID-19 from the beginning of the pandemic (February 2020) until March 2021, when he decided to resign. S.N.W. signed the Great Barrington Declaration, published 3 media articles for the ‘Spectator’ discussing tradeoffs, the evidence for when the UK infection waves peaked and the evidence for mask efficacy, a small number of media appearances advocating for sampling based assessment of incidence and prevalence and discussing tradeoffs, and the evidence on when UK infection waves peaked, and has written evidence provided to the IJK parliament Science and Technology Committee. J.P.A.I. declares that he is on the editorial board of Journal of Clinical Epidemiology and that he has published in the scientific literature both before (https://onlinelibrary.wiley.com/doi/10.1111/eci.13162) and during the pandemic (https://www.bmj.com/content/371/bmj.m4048) articles that are skeptical about the value of vote counting and signature collections for deciding scientific issues.

Figures

See this image and copyright information in PMC

References

1. Schunemann HJ, Zhang Y, Oxman AD, Expert Evidence in Guidelines G. Distinguishing opinion from evidence in guidelines. BMJ 2019;366:l4606. - PubMed
1. Djulbegovic B, Guyatt G. Evidence vs consensus in clinical practice guidelines. JAMA 2019;322:725–6. - PMC - PubMed
1. Mustafa RA, Garcia CAC, Bhatt M, Riva JJ, Vesely S, Wiercioch W, et al. GRADE notes: how to use GRADE when there is “no” evidence? A case study of the expert evidence approach. J Clin Epidemiol 2021;137:231–5. - PubMed
1. Schippers MC, Rus DC. Optimizing decision-making processes in times of Covid-19: using reflexivity to counteract information processing Failures. Front Psychol 2021;12:650525. - PMC - PubMed
1. Yao L, Ahmed MM, Guyatt GH, Yan P, Hui X, Wang Q, et al. Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis. BMJ 2021. ;375:e066045. - PMC - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

R35 GM122543/GM/NIGMS NIH HHS/United States

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Panel stacking is a threat to consensus statement validity

Affiliations

Panel stacking is a threat to consensus statement validity

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Medical