Development of a computerized adaptive substance use disorder scale for screening and measurement: the CAT-SUD

Abstract

Background and aims: The focus of this paper is on the improvement of substance use disorder (SUD) screening and measurement. Using a multi-dimensional item response theory model, the bifactor model, we provide a psychometric harmonization between SUD, depression, anxiety, trauma, social isolation, functional impairment and risk-taking behavior symptom domains, providing a more balanced view of SUD. The aims are to (1) develop the item-bank, (2) calibrate the item-bank using a bifactor model that includes a primary dimension and symptom-specific subdomains, (3) administer using computerized adaptive testing (CAT) and (4) validate the CAT-SUD in Spanish and English in the United States and Spain.

Design: Item bank construction, item calibration phase, CAT-SUD validation phase.

Setting: Primary care, community clinics, emergency departments and patient-to-patient referrals in Spain (Barcelona and Madrid) and the United States (Boston and Los Angeles).

Participants/cases: Calibration phase: the CAT-SUD was developed via simulation from complete item responses in 513 participants. Validation phase: 297 participants received the Composite International Diagnostic Interview (CIDI) and the CAT-SUD.

Measurements: A total of 252 items from five subdomains: (1) SUD, (2) psychological disorders, (3) risky behavior, (4) functional impairment and (5) social support. CAT-SUD scale scores and CIDI SUD diagnosis.

Findings: Calibration: the bifactor model provided excellent fit to the multi-dimensional item bank; 168 items had high loadings (> 0.4 with the majority > 0.6) on the primary SUD dimension. Using an average of 11 items (four to 26), which represents a 94% reduction in respondent burden (average administration time of approximately 2 minutes), we found a correlation of 0.91 with the 168-item scale (precision of 5 points on a 100-point scale).

Validation: strong agreement was found between the primary CAT-SUD dimension estimate and the results of a structured clinical interview. There was a 20-fold increase in the likelihood of a CIDI SUD diagnosis across the range of the CAT-SUD (AUC = 0.85).

Conclusions: We have developed a new approach for the screening and measurement of SUD and related severity based on multi-dimensional item response theory. The bifactor model harmonized information from mental health, trauma, social support and traditional SUD items to provide a more complete characterization of SUD. The CAT-SUD is highly predictive of a current SUD diagnosis based on a structured clinical interview, and may be predictive of the development of SUD in the future.

Keywords: Computerized adaptive testing; Latino; bifactor model; item response theory; mental health; substance use disorder.

Figure 1:

Consort Diagram

Figure 2

Predicted Probabilities of CIDI SUD Diagnosis with 95% Confidence Intervals

Figure 3:

ROC Curve