Test-retest reliability of DSM-5 substance use symptom checklists used in primary care and mental health care settings

Abstract

Introduction: Substance use disorders (SUDs) are underdiagnosed in healthcare settings. The Substance Use Symptom Checklist (SUSC) is a practical, patient-report questionnaire that has been used to assess SUD symptoms based on Diagnostic and Statistical Manual of Mental Disorders-5th edition (DSM-5) criteria. This study evaluates the test-retest reliability of SUSCs completed in primary and mental health care settings.

Methods: We identified 1194 patients who completed two SUSCs 1-21 days apart as part of routine care after reporting daily cannabis use and/or any past-year other drug use on behavioral health screens. Test-retest reliability of SUSC scores was evaluated within the full sample, subsamples who completed both checklists in primary care (n=451) or mental health clinics (n=512) where SUSC implementation differed, and subgroups defined by sex, insurance status, age, and substance use reported on behavioral health screens.

Results: In the full sample, test-retest reliability was high for indices reflecting the number of SUD symptoms endorsed (ICC=0.75, 95% CI:0.72-0.77) and DSM-5 SUD severity (kappa=0.72, 95% CI:0.69-0.75). These reliability estimates were higher in primary care (ICC=0.81, 95% CI:0.77-0.84; kappa=0.79, 95% CI:0.75-0.82, respectively) than in mental health clinics (ICC=0.74, 95% CI:0.70-0.78; kappa=0.73, 95% CI:0.68-0.77). Reliability differed by age and substance use reported on behavioral health screens, but not by sex or insurance status.

Conclusions: The SUSC has good-to-excellent test-retest reliability when completed as part of routine primary or mental health care. Symptom checklists can reliably measure symptoms consistent with DSM-5 SUD criteria, which may aid SUD-related care in primary care and mental health settings.

Keywords: Assessment; Measurement-based care; Primary care; Substance use disorder; Symptom checklist.