Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2021 Sep;30(3):e1873.
doi: 10.1002/mpr.1873. Epub 2021 May 12.

Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis

Dipika Neupane  1   2 , Brooke Levis  1   2   3 , Parash M Bhandari  1   2 , Brett D Thombs  1   2   4   5   6   7   8 , Andrea Benedetti  2   5   9 ; DEPRESsion Screening Data (DEPRESSD) Collaboration
Affiliations
Meta-Analysis

Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis

Dipika Neupane et al. Int J Methods Psychiatr Res. 2021 Sep.

Abstract

Objectives: Selectively reported results from only well-performing cutoffs in diagnostic accuracy studies may bias estimates in meta-analyses. We investigated cutoff reporting patterns for the Patient Health Questionnaire-9 (PHQ-9; standard cutoff 10) and Edinburgh Postnatal Depression Scale (EPDS; no standard cutoff, commonly used 10-13) and compared accuracy estimates based on published cutoffs versus all cutoffs.

Methods: We conducted bivariate random effects meta-analyses using individual participant data to compare accuracy from published versus all cutoffs.

Results: For the PHQ-9 (30 studies, N = 11,773), published results underestimated sensitivity for cutoffs below 10 (median difference: -0.06) and overestimated for cutoffs above 10 (median difference: 0.07). EPDS (19 studies, N = 3637) sensitivity estimates from published results were similar for cutoffs below 10 (median difference: 0.00) but higher for cutoffs above 13 (median difference: 0.14). Specificity estimates from published and all cutoffs were similar for both tools. The mean cutoff of all reported cutoffs in PHQ-9 studies with optimal cutoff below 10 was 8.8 compared to 11.8 for those with optimal cutoffs above 10. Mean for EPDS studies with optimal cutoffs below 10 was 9.9 compared to 11.8 for those with optimal cutoffs greater than 10.

Conclusion: Selective cutoff reporting was more pronounced for the PHQ-9 than EPDS.

Keywords: diagnostic test accuracy; individual participant data meta-analysis; meta-analysis; publication bias; selective cutoff reporting.

PubMed Disclaimer

Conflict of interest statement

All authors have completed the ICJME uniform disclosure form and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years with the following exceptions: Dr. Tonelli declares that he has received a grant from Merck Canada, outside the submitted work. Dr. Vigod declares that she receives royalties from UpToDate, outside the submitted work. Dr. Beck declares that she receives royalties for her Postpartum Depression Screening Scale published by Western Psychological Services. Dr. Inagaki declares that he has received a grant from Novartis Pharma, and personal fees from Meiji, Mochida, Takeda, Novartis, Yoshitomi, Pfizer, Eisai, Otsuka, MSD, Technomics, and Sumitomo Dainippon, all outside of the submitted work. Dr. Ismail declares that she has received honorarium for speaker fees for educational lectures for Sanofi, Sunovion, Janssen and Novo Nordisk. All authors declare no other relationships or activities that could appear to have influenced the submitted work. No funder had any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Figures

FIGURE 1
FIGURE 1
Receiver operating characteristic (ROC) curves plot for the diagnostic accuracy of Patient Health Questionnaire‐9 (PHQ‐9). The points in the ROC curves indicate each of the PHQ‐9 cutoffs between 5 (right) and 15 (left)
FIGURE 2
FIGURE 2
Receiver operating characteristic (ROC) curves plot for the diagnostic accuracy of Edinburgh Postnatal Depression Scale (EPDS). The points in the ROC curves indicate each of the EPDS cutoffs between 5 (right) and 18 (left)
FIGURE 3
FIGURE 3
Pattern of cutoff reporting for PHQ‐9 studies. Cells shaded in gray represent cutoff points for which diagnostic accuracy results are reported in the primary studies. “O” represents the optimal cutoff for PHQ‐9 explicitly stated in the studies except for Inagaki et al. (2013), Pence et al. (2012), Arroll (2010), Cholera (2014), Amoozegar (2017), which did not identify an optimal cutoff. For those, Youden's J optimal was calculated from published accuracies. For Gjerdingen (2009) and Vöhringer (2013), only one cutoff was reported without stating whether it was optimal or not. van Steenbergen‐Weijenburg reported 10 and 12 as optimal cutoffs. Studies that reported accuracies for cutoffs beyond presented in the table: Inagaki et al. (2013) reported the accuracy for cutoffs 4–13, Thombs (2008) reported the accuracy for cutoffs 1–10, Lambert et al. (2015) reported the accuracy for cutoffs 5, 9, 10, 15, 20, Hyphantis (2011) reported the accuracy for cutoffs 4–16, Osorio (2009) reported the accuracy for cutoffs 10–21. All the reported cutoffs were included while calculating the mean of reported cutoffs though they are not shown in the figure
FIGURE 4
FIGURE 4
Pattern of cutoff reporting for EPDS studies. Cells shaded in gray represent cutoff points for which diagnostic accuracy results are reported in the primary studies. “O” represents the optimal cutoff for EPDS explicitly stated in the studies except for Philips (2009), which did not identify an optimal cutoff. For Philips , Youden's J optimal was calculated from published accuracies. For Bakare et al. (2014), Pawlby et al. (2008), Beck  only one cutoff was reported without stating whether it was optimal or not. Studies that reported accuracies for cutoffs beyond presented in the table: Khalifa et al. (2015) reported accuracy for cutoffs 1–15, Vega‐Dienstmaier et al. (2002) reported the accuracy for cutoffs 1–26. All the reported cutoffs were included while calculating the mean of reported cutoffs though they are not shown in the figure
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Alvarado, R., Jadresic, E., Guajardo, V., & Rojas, G. (2015). First validation of a Spanish‐translated version of the Edinburgh Postnatal Depression Scale (EPDS) for use in pregnant women. A Chilean study. Archives of Women's Mental Health, 18(4), 607–612. 10.1007/s00737-014-0466-z - DOI - PubMed
    1. Amoozegar, F., Patten, S. B., Becker, W. J., Bulloch, A. G. M., Fiest, K. M., Davenport, W. J., Carroll, C. R., & Jette, N. (2017). The prevalence of depression and the accuracy of depression screening tools in migraine patients. General Hospital Psychiatry, 48, 25–31. 10.1016/j.genhosppsych.2017.06.006 - DOI - PubMed
    1. Arroll, B., Goodyear‐Smith, F., Crengle, S., Gunn, J., Kerse, N., Fishman, T., Falloon, K., & Hatcher, S. (2010). Validation of PHQ‐2 and PHQ‐9 to screen for major depression in the primary care population. The Annals of Family Medicine, 8(4), 348–353. 10.1370/afm.1139 - DOI - PMC - PubMed
    1. Bakare, M. O., Okoye, J. O., & Obindo, J. T. (2014). Introducing depression and developmental screenings into the national programme on immunization (NPI) in southeast Nigeria: An experimental cross‐sectional assessment. General Hospital Psychiatry, 36(1), 105–112. 10.1016/j.genhosppsych.2013.09.005 - DOI - PubMed
    1. Beck, C. T., & Gable, R. K. (2001). Comparative analysis of the performance of the postpartum depression screening scale with two other depression instruments. Nursing Research, 50(4), 242–250. 10.1097/00006199-200107000-00008 - DOI - PubMed

Publication types