. 2024 Nov 4;7(11):e2429630.

doi: 10.1001/jamanetworkopen.2024.29630.

Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool

Brooke Levis^{1

2}, Parash Mani Bhandari¹, Dipika Neupane¹, Suiqiong Fan¹, Ying Sun¹, Chen He¹, Yin Wu¹, Ankur Krishnan¹, Zelalem Negeri³, Mahrukh Imran¹, Danielle B Rice⁴, Kira E Riehm¹, Marleine Azar¹, Alexander W Levis¹, Jill Boruff⁵, Pim Cuijpers⁶, Simon Gilbody⁷, John P A Ioannidis^{8

9

10

11}, Lorie A Kloda¹², Scott B Patten¹³, Roy C Ziegelstein¹⁴, Daphna Harel¹⁵, Yemisi Takwoingi¹⁶, Sarah Markham¹⁷, Sultan H Alamri¹⁸, Dagmar Amtmann¹⁹, Bruce Arroll²⁰, Liat Ayalon²¹, Hamid R Baradaran²², Anna Beraldi²³, Charles N Bernstein²⁴, Arvin Bhana²⁵, Charles H Bombardier¹⁹, Ryna Imma Buji²⁶, Peter Butterworth²⁷, Gregory Carter²⁸, Marcos H Chagas²⁹, Juliana C N Chan³⁰, Lai Fong Chan³¹, Dixon Chibanda³², Kerrie Clover³³, Aaron Conway³⁴, Yeates Conwell³⁵, Federico M Daray³⁶, Janneke M de Man-van Ginkel³⁷, Jesse R Fann³⁸, Felix H Fischer³⁹, Sally Field⁴⁰, Jane R W Fisher⁴¹, Daniel S S Fung⁴², Bizu Gelaye⁴³, Leila Gholizadeh⁴⁴, Felicity Goodyear-Smith²⁰, Eric P Green⁴⁵, Catherine G Greeno⁴⁶, Brian J Hall⁴⁷, Liisa Hantsoo⁴⁸, Martin Härter⁴⁹, Leanne Hides⁵⁰, Stevan E Hobfoll⁵¹, Simone Honikman⁴⁰, Thomas Hyphantis⁵², Masatoshi Inagaki⁵³, Maria Iglesias-Gonzalez⁵⁴, Hong Jin Jeon⁵⁵, Nathalie Jetté⁵⁶, Mohammad E Khamseh²², Kim M Kiely⁵⁷, Brandon A Kohrt⁵⁸, Yunxin Kwan⁵⁹, Maria Asunción Lara⁶⁰, Holly F Levin-Aspenson⁶¹, Shen-Ing Liu⁶², Manote Lotrakul⁶³, Sonia R Loureiro²⁹, Bernd Löwe⁶⁴, Nagendra P Luitel⁶⁵, Crick Lund⁶⁶, Ruth Ann Marrie⁶⁷, Laura Marsh⁶⁸, Brian P Marx⁶⁹, Anthony McGuire⁷⁰, Sherina Mohd Sidik⁷¹, Tiago N Munhoz⁷², Kumiko Muramatsu⁷³, Juliet E M Nakku⁷⁴, Laura Navarrete⁷⁵, Flávia L Osório²⁹, Brian W Pence⁷⁶, Philippe Persoons⁷⁷, Inge Petersen²⁵, Angelo Picardi⁷⁸, Stephanie L Pugh⁷⁹, Terence J Quinn⁸⁰, Elmars Rancans^{80

81}, Sujit D Rathod⁸², Katrin Reuter⁸³, Alasdair G Rooney⁸⁴, Iná S Santos⁷², Miranda T Schram⁸⁵, Juwita Shaaban⁸⁶, Eileen H Shinn⁸⁷, Abbey Sidebottom⁸⁸, Adam Simning³⁵, Lena Spangenberg⁸⁹, Lesley Stafford⁹⁰, Sharon C Sung⁶², Keiko Suzuki⁹¹, Pei Lin Lynnette Tan⁵⁹, Martin Taylor-Rowan⁹², Thach D Tran⁴¹, Alyna Turner⁹³, Christina M van der Feltz-Cornelis⁹⁴, Thandi van Heyningen⁹⁵, Paul A Vöhringer⁹⁶, Lynne I Wagner⁹⁷, Jian Li Wang⁹⁸, David Watson⁹⁹, Jennifer White¹⁰⁰, Mary A Whooley^{101

102}, Kirsty Winkley¹⁰³, Karen Wynter¹⁰⁴, Mitsuhiko Yamada¹⁰⁵, Qing Zhi Zeng¹⁰⁶, Yuying Zhang¹⁰⁷, Brett D Thombs^{1

2

108

109

110

111}, Andrea Benedetti^{2

109

112

113}; Depression Screening Data (DEPRESSD) PHQ Group

Affiliations

¹ Lady Davis Institute for Medical Research, Jewish General Hospital, Montréal, Québec, Canada.
² Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Québec, Canada.
³ Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada.
⁴ Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada.
⁵ Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montréal, Québec, Canada.
⁶ Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.
⁷ Hull York Medical School and the Department of Health Sciences, University of York, Heslington, York, UK.
⁸ Department of Medicine, Stanford University, Stanford, California.
⁹ Department of Epidemiology and Population Health, Stanford University, Stanford, California.
¹⁰ Department of Biomedical Data Science,Stanford University, Stanford, California.
¹¹ Department of Statistics, Stanford University, Stanford, California.
¹² McGill University Libraries, Montréal, Québec, Canada.
¹³ Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.
¹⁴ Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland.
¹⁵ Department of Applied Statistics, Social Science, and Humanities, New York University, New York.
¹⁶ Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK.
¹⁷ Department of Biostatistics and Health Informatics, King's College London, London, UK.
¹⁸ Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia.
¹⁹ Department of Rehabilitation Medicine, University of Washington, Seattle.
²⁰ Department of General Practice and Primary Health Care, University of Auckland, Auckland, New Zealand.
²¹ Louis and Gabi Weisfeld School of Social Work, Bar Ilan University, Ramat Gan, Israel.
²² Endocrine Research Center, Institute of Endocrinology and Metabolism, Iran University of Medical Sciences, Tehran, Iran.
²³ Kbo-Lech-Mangfall-Klinik Garmisch-Partenkirchen, Klinik für Psychiatrie, Psychotherapie and Psychosomatik, Lehrkrankenhaus der Technischen Universität München, Munich, Germany.
²⁴ University of Manitoba IBD Clinical and Research Centre, Winnipeg, Manitoba, Canada.
²⁵ Centre for Rural Health, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, KwaZulu-Natal, South Africa.
²⁶ Department of Psychiatry, Hospital Mesra Bukit Padang, Sabah, Malaysia.
²⁷ Centre for Epidemiology and Population Health, The Australian National University, Canberra, Australia.
²⁸ Centre for Brain and Mental Health Research, University of Newcastle, Newcastle, New South Wales, Australia.
²⁹ Department of Neurosciences and Behavior, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, São Paulo, Brazil.
³⁰ Department of Medicine and Therapeutics, Hong Kong Institute of Diabetes and Obesity and Li Ka Shing Institute of Health Science, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong Special Administrative Region, China.
³¹ Department of Psychiatry, National University of Malaysia, Kuala Lumpur, Malaysia.
³² Department of Community Medicine, University of Zimbabwe, Harare, Zimbabwe.
³³ School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia.
³⁴ School of Nursing, Queensland University of Technology, Brisbane, Queensland, Australia.
³⁵ Department of Psychiatry, University of Rochester Medical Center, Rochester, New York.
³⁶ Institute of Pharmacology, School of Medicine, University of Buenos Aires, Buenos Aires, Argentina.
³⁷ Leids University Medical Center, Leiden, the Netherlands.
³⁸ Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle.
³⁹ Center for Patient-Centered Outcomes Research, Department of Psychosomatic Medicine, Center for Internal Medicine and Dermatology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany.
⁴⁰ Perinatal Mental Health Project, Alan J Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa.
⁴¹ Global and Women's Health, Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
⁴² Department of Developmental Psychiatry, Institute of Mental Health, Singapore.
⁴³ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
⁴⁴ Faculty of Health, University of Technology Sydney, Sydney, New South Wales, Australia.
⁴⁵ Duke Global Health Institute, Duke University, Durham, North Carolina.
⁴⁶ School of Social Work, University of Pittsburgh, Pittsburgh, Pennsylvania.
⁴⁷ Center for Global Health Equity, New York University Shanghai, Shanghai, China.
⁴⁸ Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University School of Medicine, Baltimore, Maryland.
⁴⁹ Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
⁵⁰ School of Psychology, The University of Queensland, Brisbane, Queensland, Australia.
⁵¹ STAR-Stress, Anxiety and Resilience Consultants, Chicago, Illinois.
⁵² Department of Psychiatry, Faculty of Medicine, School of Health Sciences, University of Ioannina, Ioannina, Greece.
⁵³ Department of Psychiatry, Faculty of Medicine, Shimane University, Izumo, Shimane, Japan.
⁵⁴ Department of Psychiatry, Hospital Universitari Germans Trias i Pujol, Badalona, Spain.
⁵⁵ Department of Psychiatry, Depression Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea.
⁵⁶ Department of Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada.
⁵⁷ School of Health and Society and School of Mathematics and Applied Statistics, University of Wollongong, Wollongong, New South Wales, Australia.
⁵⁸ Center for Global Mental Health Equity, The George Washington University, Washington, DC.
⁵⁹ Department of Psychological Medicine, Tan Tock Seng Hospital, Singapore.
⁶⁰ Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz, San Lorenzo Huipulco, Tlalpan, Mexico.
⁶¹ Department of Psychology, University of North Texas, Denton.
⁶² Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore.
⁶³ Department of Psychiatry, Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand.
⁶⁴ Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
⁶⁵ Research Department, TPO Nepal, Kathmandu, Nepal.
⁶⁶ Centre for Global Mental Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
⁶⁷ Departments of Medicine and Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada.
⁶⁸ Baylor College of Medicine, Houston and Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas.
⁶⁹ National Center for PTSD at Veterans Affairs Boston Healthcare System, Boston, Massachusetts.
⁷⁰ College of Nursing, University of South Florida, Tampa.
⁷¹ Department of Psychiatry, Faculty of Medicine and Health Sciences, Universiti Putra Malaysia, Serdang, Selangor, Malaysia.
⁷² Post-Graduate Program in Epidemiology, Federal University of Pelotas, Pelotas, Rio Grande do Sul, Brazil.
⁷³ Niigata Seiryo University Health Service Center, Niigata, Japan.
⁷⁴ Butabika National Referral Teaching Hospital, Kampala, Uganda.
⁷⁵ Department of Epidemiology and Psychosocial Research, Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz, Ciudad de México, México.
⁷⁶ Department of Epidemiology, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill.
⁷⁷ Department of Psycho-Pedagogic Psychiatry, Healthcare Group Sint-Kamillus, Broeders van Liefde, Bierbeek, Belgium.
⁷⁸ Centre for Behavioural Sciences and Mental Health, Italian National Institute of Health, Rome, Italy.
⁷⁹ Department of Statistics, American College of Radiology, NRG Oncology Statistics and Data Management Center, Philadelphia, Pennsylvania.
⁸⁰ Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, Scotland, UK.
⁸¹ Department of Psychiatry and Narcology, Riga Stradins University, Riga, Latvia.
⁸² Department of Population Health, London School of Hygiene and Tropical Medicine, London, UK.
⁸³ Group Practice for Psychotherapy and Psycho-oncology, Freiburg, Germany.
⁸⁴ Division of Psychiatry, Royal Edinburgh Hospital, The University of Edinburgh, Edinburgh, Scotland, UK.
⁸⁵ Department of Internal Medicine, Maastricht University Medical Center, Maastricht, the Netherlands.
⁸⁶ Department of Family Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kelantan, Malaysia.
⁸⁷ Department of Behavioral Science, The University of Texas M.D. Anderson Cancer Center, Houston.
⁸⁸ Allina Health, Minneapolis, Minnesota.
⁸⁹ Department of Medical Psychology and Medical Sociology, University of Leipzig, Leipzig, Germany.
⁹⁰ Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Victoria, Australia.
⁹¹ Department of General Medicine, Asahikawa University Hospital, Asahikawa, Hokkaido, Japan.
⁹² Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, UK.
⁹³ IMPACT-the Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Deakin University, Geelong, Victoria, Australia.
⁹⁴ Department of Health Sciences, Hull York Medical School, University of York, York, UK.
⁹⁵ Justice and Violence Prevention Programme, Institute for Security Studies, Pretoria, South Africa.
⁹⁶ Department of Psychiatry and Mental Health, Clinical Hospital, Universidad de Chile, Santiago, Chile.
⁹⁷ Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill.
⁹⁸ Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada.
⁹⁹ Department of Psychology, University of Notre Dame, Notre Dame, Indiana.
¹⁰⁰ School of Medicine and Public Health, College of Health, Medicine and Wellbeing, University of Newcastle, New South Wales, Australia.
¹⁰¹ Department of Medicine, University of California San Francisco, San Francisco.
¹⁰² Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco.
¹⁰³ Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care, King's College London, London, UK.
¹⁰⁴ School of Clinical Sciences, Monash University, Melbourne, Victoria, Australia.
¹⁰⁵ Department of Pathophysiology, Tokyo Kasei Gakuin University, Chiyoda-ku, Tokyo, Japan.
¹⁰⁶ Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
¹⁰⁷ Department of Medicine and Therapeutics, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China.
¹⁰⁸ Department of Psychiatry, McGill University, Montréal, Québec, Canada.
¹⁰⁹ Department of Medicine, McGill University, Montréal, Québec, Canada.
¹¹⁰ Department of Psychology, McGill University, Montréal, Québec, Canada.
¹¹¹ Biomedical Ethics Unit, McGill University, Montréal, Québec, Canada.
¹¹² Respiratory Epidemiology and Clinical Research Unit, McGill University Health Centre, Montréal, Québec, Canada.
¹¹³ Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montréal, Québec, Canada.

PMID: 39576645
PMCID: PMC11584932
DOI: 10.1001/jamanetworkopen.2024.29630

Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool

Brooke Levis et al. JAMA Netw Open. 2024.

. 2024 Nov 4;7(11):e2429630.

doi: 10.1001/jamanetworkopen.2024.29630.

Authors

Affiliations

¹ Lady Davis Institute for Medical Research, Jewish General Hospital, Montréal, Québec, Canada.
² Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Québec, Canada.
³ Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada.
⁴ Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada.
⁵ Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montréal, Québec, Canada.
⁶ Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands.
⁷ Hull York Medical School and the Department of Health Sciences, University of York, Heslington, York, UK.
⁸ Department of Medicine, Stanford University, Stanford, California.
⁹ Department of Epidemiology and Population Health, Stanford University, Stanford, California.
¹⁰ Department of Biomedical Data Science,Stanford University, Stanford, California.
¹¹ Department of Statistics, Stanford University, Stanford, California.
¹² McGill University Libraries, Montréal, Québec, Canada.
¹³ Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.
¹⁴ Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland.
¹⁵ Department of Applied Statistics, Social Science, and Humanities, New York University, New York.
¹⁶ Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK.
¹⁷ Department of Biostatistics and Health Informatics, King's College London, London, UK.
¹⁸ Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia.
¹⁹ Department of Rehabilitation Medicine, University of Washington, Seattle.
²⁰ Department of General Practice and Primary Health Care, University of Auckland, Auckland, New Zealand.
²¹ Louis and Gabi Weisfeld School of Social Work, Bar Ilan University, Ramat Gan, Israel.
²² Endocrine Research Center, Institute of Endocrinology and Metabolism, Iran University of Medical Sciences, Tehran, Iran.
²³ Kbo-Lech-Mangfall-Klinik Garmisch-Partenkirchen, Klinik für Psychiatrie, Psychotherapie and Psychosomatik, Lehrkrankenhaus der Technischen Universität München, Munich, Germany.
²⁴ University of Manitoba IBD Clinical and Research Centre, Winnipeg, Manitoba, Canada.
²⁵ Centre for Rural Health, School of Nursing and Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban, KwaZulu-Natal, South Africa.
²⁶ Department of Psychiatry, Hospital Mesra Bukit Padang, Sabah, Malaysia.
²⁷ Centre for Epidemiology and Population Health, The Australian National University, Canberra, Australia.
²⁸ Centre for Brain and Mental Health Research, University of Newcastle, Newcastle, New South Wales, Australia.
²⁹ Department of Neurosciences and Behavior, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, São Paulo, Brazil.
³⁰ Department of Medicine and Therapeutics, Hong Kong Institute of Diabetes and Obesity and Li Ka Shing Institute of Health Science, The Chinese University of Hong Kong, Prince of Wales Hospital, Shatin, Hong Kong Special Administrative Region, China.
³¹ Department of Psychiatry, National University of Malaysia, Kuala Lumpur, Malaysia.
³² Department of Community Medicine, University of Zimbabwe, Harare, Zimbabwe.
³³ School of Medicine and Public Health, University of Newcastle, Callaghan, New South Wales, Australia.
³⁴ School of Nursing, Queensland University of Technology, Brisbane, Queensland, Australia.
³⁵ Department of Psychiatry, University of Rochester Medical Center, Rochester, New York.
³⁶ Institute of Pharmacology, School of Medicine, University of Buenos Aires, Buenos Aires, Argentina.
³⁷ Leids University Medical Center, Leiden, the Netherlands.
³⁸ Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle.
³⁹ Center for Patient-Centered Outcomes Research, Department of Psychosomatic Medicine, Center for Internal Medicine and Dermatology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany.
⁴⁰ Perinatal Mental Health Project, Alan J Flisher Centre for Public Mental Health, Department of Psychiatry and Mental Health, University of Cape Town, Cape Town, South Africa.
⁴¹ Global and Women's Health, Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
⁴² Department of Developmental Psychiatry, Institute of Mental Health, Singapore.
⁴³ Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts.
⁴⁴ Faculty of Health, University of Technology Sydney, Sydney, New South Wales, Australia.
⁴⁵ Duke Global Health Institute, Duke University, Durham, North Carolina.
⁴⁶ School of Social Work, University of Pittsburgh, Pittsburgh, Pennsylvania.
⁴⁷ Center for Global Health Equity, New York University Shanghai, Shanghai, China.
⁴⁸ Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University School of Medicine, Baltimore, Maryland.
⁴⁹ Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
⁵⁰ School of Psychology, The University of Queensland, Brisbane, Queensland, Australia.
⁵¹ STAR-Stress, Anxiety and Resilience Consultants, Chicago, Illinois.
⁵² Department of Psychiatry, Faculty of Medicine, School of Health Sciences, University of Ioannina, Ioannina, Greece.
⁵³ Department of Psychiatry, Faculty of Medicine, Shimane University, Izumo, Shimane, Japan.
⁵⁴ Department of Psychiatry, Hospital Universitari Germans Trias i Pujol, Badalona, Spain.
⁵⁵ Department of Psychiatry, Depression Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea.
⁵⁶ Department of Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada.
⁵⁷ School of Health and Society and School of Mathematics and Applied Statistics, University of Wollongong, Wollongong, New South Wales, Australia.
⁵⁸ Center for Global Mental Health Equity, The George Washington University, Washington, DC.
⁵⁹ Department of Psychological Medicine, Tan Tock Seng Hospital, Singapore.
⁶⁰ Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz, San Lorenzo Huipulco, Tlalpan, Mexico.
⁶¹ Department of Psychology, University of North Texas, Denton.
⁶² Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore.
⁶³ Department of Psychiatry, Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand.
⁶⁴ Department of Psychosomatic Medicine and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
⁶⁵ Research Department, TPO Nepal, Kathmandu, Nepal.
⁶⁶ Centre for Global Mental Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
⁶⁷ Departments of Medicine and Community Health Sciences, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada.
⁶⁸ Baylor College of Medicine, Houston and Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas.
⁶⁹ National Center for PTSD at Veterans Affairs Boston Healthcare System, Boston, Massachusetts.
⁷⁰ College of Nursing, University of South Florida, Tampa.
⁷¹ Department of Psychiatry, Faculty of Medicine and Health Sciences, Universiti Putra Malaysia, Serdang, Selangor, Malaysia.
⁷² Post-Graduate Program in Epidemiology, Federal University of Pelotas, Pelotas, Rio Grande do Sul, Brazil.
⁷³ Niigata Seiryo University Health Service Center, Niigata, Japan.
⁷⁴ Butabika National Referral Teaching Hospital, Kampala, Uganda.
⁷⁵ Department of Epidemiology and Psychosocial Research, Instituto Nacional de Psiquiatría Ramón de la Fuente Muñiz, Ciudad de México, México.
⁷⁶ Department of Epidemiology, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill.
⁷⁷ Department of Psycho-Pedagogic Psychiatry, Healthcare Group Sint-Kamillus, Broeders van Liefde, Bierbeek, Belgium.
⁷⁸ Centre for Behavioural Sciences and Mental Health, Italian National Institute of Health, Rome, Italy.
⁷⁹ Department of Statistics, American College of Radiology, NRG Oncology Statistics and Data Management Center, Philadelphia, Pennsylvania.
⁸⁰ Institute of Cardiovascular and Medical Sciences, University of Glasgow, Glasgow, Scotland, UK.
⁸¹ Department of Psychiatry and Narcology, Riga Stradins University, Riga, Latvia.
⁸² Department of Population Health, London School of Hygiene and Tropical Medicine, London, UK.
⁸³ Group Practice for Psychotherapy and Psycho-oncology, Freiburg, Germany.
⁸⁴ Division of Psychiatry, Royal Edinburgh Hospital, The University of Edinburgh, Edinburgh, Scotland, UK.
⁸⁵ Department of Internal Medicine, Maastricht University Medical Center, Maastricht, the Netherlands.
⁸⁶ Department of Family Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kelantan, Malaysia.
⁸⁷ Department of Behavioral Science, The University of Texas M.D. Anderson Cancer Center, Houston.
⁸⁸ Allina Health, Minneapolis, Minnesota.
⁸⁹ Department of Medical Psychology and Medical Sociology, University of Leipzig, Leipzig, Germany.
⁹⁰ Melbourne School of Psychological Sciences, The University of Melbourne, Melbourne, Victoria, Australia.
⁹¹ Department of General Medicine, Asahikawa University Hospital, Asahikawa, Hokkaido, Japan.
⁹² Institute of Health and Wellbeing, University of Glasgow, Glasgow, Scotland, UK.
⁹³ IMPACT-the Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Deakin University, Geelong, Victoria, Australia.
⁹⁴ Department of Health Sciences, Hull York Medical School, University of York, York, UK.
⁹⁵ Justice and Violence Prevention Programme, Institute for Security Studies, Pretoria, South Africa.
⁹⁶ Department of Psychiatry and Mental Health, Clinical Hospital, Universidad de Chile, Santiago, Chile.
⁹⁷ Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, Chapel Hill.
⁹⁸ Department of Community Health and Epidemiology, Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada.
⁹⁹ Department of Psychology, University of Notre Dame, Notre Dame, Indiana.
¹⁰⁰ School of Medicine and Public Health, College of Health, Medicine and Wellbeing, University of Newcastle, New South Wales, Australia.
¹⁰¹ Department of Medicine, University of California San Francisco, San Francisco.
¹⁰² Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco.
¹⁰³ Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care, King's College London, London, UK.
¹⁰⁴ School of Clinical Sciences, Monash University, Melbourne, Victoria, Australia.
¹⁰⁵ Department of Pathophysiology, Tokyo Kasei Gakuin University, Chiyoda-ku, Tokyo, Japan.
¹⁰⁶ Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
¹⁰⁷ Department of Medicine and Therapeutics, Prince of Wales Hospital, The Chinese University of Hong Kong, Hong Kong Special Administrative Region, China.
¹⁰⁸ Department of Psychiatry, McGill University, Montréal, Québec, Canada.
¹⁰⁹ Department of Medicine, McGill University, Montréal, Québec, Canada.
¹¹⁰ Department of Psychology, McGill University, Montréal, Québec, Canada.
¹¹¹ Biomedical Ethics Unit, McGill University, Montréal, Québec, Canada.
¹¹² Respiratory Epidemiology and Clinical Research Unit, McGill University Health Centre, Montréal, Québec, Canada.
¹¹³ Centre for Outcomes Research and Evaluation, Research Institute of the McGill University Health Centre, Montréal, Québec, Canada.

PMID: 39576645
PMCID: PMC11584932
DOI: 10.1001/jamanetworkopen.2024.29630

Abstract

Importance: Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates.

Objective: To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimate accuracy yields (1) optimal cutoff scores that differ from the population-level optimal cutoff score and (2) biased accuracy estimates.

Design, setting, and participants: This study used cross-sectional data from an existing individual participant data meta-analysis (IPDMA) database on PHQ-9 screening accuracy to represent a hypothetical population. Studies in the IPDMA database compared participant PHQ-9 scores with a major depression classification. From the IPDMA population, 1000 studies of 100, 200, 500, and 1000 participants each were resampled.

Main outcomes and measures: For the full IPDMA population and each simulated study, an optimal cutoff score was selected by maximizing the Youden index. Accuracy estimates for optimal cutoff scores in simulated studies were compared with accuracy in the full population.

Results: The IPDMA database included 100 primary studies with 44 503 participants (4541 [10%] cases of major depression). The population-level optimal cutoff score was 8 or higher. Optimal cutoff scores in simulated studies ranged from 2 or higher to 21 or higher in samples of 100 participants and 5 or higher to 11 or higher in samples of 1000 participants. The percentage of simulated studies that identified the true optimal cutoff score of 8 or higher was 17% for samples of 100 participants and 33% for samples of 1000 participants. Compared with estimates for a cutoff score of 8 or higher in the population, sensitivity was overestimated by 6.4 (95% CI, 5.7-7.1) percentage points in samples of 100 participants, 4.9 (95% CI, 4.3-5.5) percentage points in samples of 200 participants, 2.2 (95% CI, 1.8-2.6) percentage points in samples of 500 participants, and 1.8 (95% CI, 1.5-2.1) percentage points in samples of 1000 participants. Specificity was within 1 percentage point across sample sizes.

Conclusions and relevance: This study of cross-sectional data found that optimal cutoff scores and accuracy estimates differed substantially from population values when data-driven methods were used to simultaneously identify an optimal cutoff score and estimate accuracy. Users of diagnostic accuracy evidence should evaluate studies of accuracy with caution and ensure that cutoff score recommendations are based on adequately powered research or well-conducted meta-analyses.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Disclosures: Dr Ayalon reported receiving grants from Lundbeck during the conduct of the study. Dr Bernstein reported receiving grants from AbbVie, Eli Lilly, Fresenius Kabi, Janssen, Pfizer, Takeda, Boston Scientific, JAMP Pharma, Organon, and Sandoz and personal fees from AbbVie, Amgen, Bristol Myers Squibb, Eli Lilly, Fresenius Kabi, Janssen, Pfizer, and Takeda outside the submitted work. Dr Butterworth reported receiving grants from Safe Work Australia and Australian Research Council during the conduct of the study. Dr J.C.N. Chan reported receiving grants from the European Foundation for Study of Diabetes during the conduct of the study. Dr L.F. Chan reported receiving nonfinancial support from Otsuka and Lundbeck and personal fees from Johnson & Johnson during the conduct of the study and nonfinancial support from Ortho-McNeil-Janssen and Menarini outside the submitted work. Dr Inagaki reported receiving personal fees from Meiji, Mochida, Takeda, Novartis, Yoshitomi, Pfizer, Eisai, Otsuka, MSD, Sumitomo Dainippon, Janssen, and Eli Lilly outside the submitted work. Dr Rancans reported receiving grants from Gedeon Richter; personal fees and nonfinancial support from Gedeon Richter, Lundbeck, Servier, and Janssen Cilag; and personal fees from Zentiva and AbbVie outside the submitted work. Dr Shinn reported receiving grants from the National Cancer Institute (NCI) during the conduct of the study. Dr Simning reported receiving grants from the Agency for Healthcare Research and Quality, National Center for Research Resources, and National Institute of General Medical Sciences during the conduct of the study. Dr Stafford reported receiving a PhD scholarship from The University of Melbourne during the conduct of the study. Dr Wagner reported receiving grants from the State of Pennsylvania tobacco settlement fund and NCI during the conduct of the study and personal fees from Celgene/Bristol Myers Squibb outside the submitted work. Dr Benedetti reported receiving grants from the Canadian Institutes of Health Research (CIHR) during the conduct of the study. No other disclosures were reported.

Figures

**Figure 1.. Variability of Data-Driven Optimal Cutoff Scores in 1000 Resampled Studies of 100, 200, 500, and 1000 Participants**

Figure 2.. Variability in Accuracy Estimates of the Optimal Cutoff Scores in 1000 Resampled Studies of 100, 200, 500, and 1000 Participants vs Accuracy Values for a Cutoff of 8 or Higher in the Population
Edges of boxes represent the 25th and 75th percentiles; horizontal line inside boxes represents the median; dashed horizontal line represents the accuracy of the true population-level optimal cutoff score in the full Patient Health Questionnaire-9 individual participant data meta-analysis dataset (cutoff score ≥8; sensitivity = 80.4%, specificity = 82.0%); and dots represent outliers.

Figure 3.. Variability in Accuracy Estimates of a Cutoff Score of 8 or Higher in 1000 Resampled Studies of 100, 200, 500, and 1000 Participants vs Accuracy Values for a Cutoff of 8 or Higher in the Population
Edges of boxes represent the 25th and 75th percentiles; horizontal line inside boxes represents the median; dashed horizontal line represents the accuracy of the true population-level optimal cutoff score in the full Patient Health Questionnaire-9 individual participant data meta-analysis dataset (cutoff score ≥8; sensitivity = 80.4%, specificity = 82.0%); and dots represent outliers.

See this image and copyright information in PMC

References

1. Brehaut E, Neupane D, Levis B, et al. . ‘Optimal’ cutoff selection in studies of depression screening tool accuracy using the PHQ-9, EPDS, or HADS-D: a meta-research study. Int J Methods Psychiatr Res. 2023;32(3):e1956. doi:10.1002/mpr.1956 - DOI - PMC - PubMed
1. Thombs BD, Rice DB. Sample sizes and precision of estimates of sensitivity and specificity from primary studies on the diagnostic accuracy of depression screening tools: a survey of recently published studies. Int J Methods Psychiatr Res. 2016;25(2):145-152. doi:10.1002/mpr.1504 - DOI - PMC - PubMed
1. Nassar EL, Levis B, Neyer MA, et al. . Sample size and precision of estimates in studies of depression screening tool accuracy: a meta-research review of studies published in 2018-2021. Int J Methods Psychiatr Res. 2022;31(2):e1910. doi:10.1002/mpr.1910 - DOI - PMC - PubMed
1. Linnet K, Brandt E. Assessing diagnostic tests once an optimal cutoff point has been selected. Clin Chem. 1986;32(7):1341-1346. doi:10.1093/clinchem/32.7.1341 - DOI - PubMed
1. Ewald B. Post hoc choice of cut points introduced bias to diagnostic research. J Clin Epidemiol. 2006;59(8):798-801. doi:10.1016/j.jclinepi.2005.11.025 - DOI - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool

Affiliations

Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources