A Comparison of Content-Balancing Procedures for Estimating Multiple Clinical Domains in Computerized Adaptive Testing: Relative Precision, Validity, and Detection of Persons With Misfitting Responses

Abstract

This simulation study sought to compare four different computerized adaptive testing (CAT) content balancing procedures designed for use in a multidimensional assessment with respect to measurement precision, symptom severity classification; validity of clinical diagnostic recommendations; and sensitivity to atypical responding. The four content balancing procedures were: (1) no content balancing, (2) Screener-based, (3) Mixed (Screener plus content balancing), and (4) Full content balancing. In Full content balancing and in Mixed content balancing following administration of the screener items, item selection was based on: (1) whether the target number of items for the item's subscale was reached, and (2) the item's information function. Mixed and Full content balancing provided the best representation of items from each of the main IMDS subscales. These procedures also resulted in higher CAT-to-full-scale correlations for the Trauma and Homicidal/Suicidal Thought subscales, and improved detection of atypical responding.

Keywords: Global Appraisal of Individual Needs (GAIN); Rasch measurement; atypical responding; classification; computerized adaptive testing; content balancing; diagnostic classification; multidimensional assessment; substance use disorder.