An Examination of Racial Bias in Scoring the Autism Diagnostic Observation Schedule (ADOS) Module 3: An Item Response Theory Analysis

Abstract

Given the rising prevalence of autism among racial minority children in the United States, but persistent service use disparities, this study examines potential bias in specific items from the autism diagnostic observation schedule (ADOS), a highly regarded autism evaluation. We leveraged unidimensional item response theory graded response models and a sample of 735 children to analyze the differential item functioning (DIF) of items within ADOS Module 3. Three items showed significant signs of racial bias: A1 (overall language level), A5 (offers information), and D5 (compulsions and rituals). On these items, Black/African American and Asian children were usually more likely to be rated as showing autistic behaviors than White children with similar autism levels. The impact of racial bias on the item score was small, and the impact on the overall test score was even smaller: on a scale of 0-48 points, the effect of racial bias was estimated at 0.23 total points for Black/African American children and 0.16 points for Asian children. Furthermore, none of the items showing significant bias contribute to the autism classification algorithm. This analysis suggests a small but detectable amount of bias in several specific ADOS items, but not in items central to informing an autism diagnosis. Thus, bias appears statistically, but not clinically, significant. This contributes to examinations of racial bias in the ADOS as the first analysis of Asian children and the first in-depth look at all items in the most commonly used version among school-aged children.

Keywords: ADOS; autism; differential item functioning; item response theory; racial bias.