Using computerized adaptive testing to reduce the burden of mental health assessment

Abstract

Objective: This study investigated the combination of item response theory and computerized adaptive testing (CAT) for psychiatric measurement as a means of reducing the burden of research and clinical assessments.

Methods: Data were from 800 participants in outpatient treatment for a mood or anxiety disorder; they completed 616 items of the 626-item Mood and Anxiety Spectrum Scales (MASS) at two times. The first administration was used to design and evaluate a CAT version of the MASS by using post hoc simulation. The second confirmed the functioning of CAT in live testing.

Results: Tests of competing models based on item response theory supported the scale's bifactor structure, consisting of a primary dimension and four group factors (mood, panic-agoraphobia, obsessive-compulsive, and social phobia). Both simulated and live CAT showed a 95% average reduction (585 items) in items administered (24 and 30 items, respectively) compared with administration of the full MASS. The correlation between scores on the full MASS and the CAT version was .93. For the mood disorder subscale, differences in scores between two groups of depressed patients--one with bipolar disorder and one without--on the full scale and on the CAT showed effect sizes of .63 (p<.003) and 1.19 (p<.001) standard deviation units, respectively, indicating better discriminant validity for CAT.

Conclusions: Instead of using small fixed-length tests, clinicians can create item banks with a large item pool, and a small set of the items most relevant for a given individual can be administered with no loss of information, yielding a dramatic reduction in administration time and patient and clinician burden.

Figure 1

Item-by-item administration to a study participant of the Mood and Anxiety Spectrum Scales by use of computerized adaptive testing^{a a} Possible impairment scores range from −4 to 4, with higher scores indicating greater impairment. The vertical bars indicate a standard error band of 1. Final impairment estimate: M±SE=1.26±.30

Figure 2

Frequency distribution of impairment estimates for the entire calibration sample (N=800) ^a Possible impairment scores range from −4 to 4, with higher scores indicating greater impairment.