A Critical Appraisal of the Measurement of Adaptive Social Communication Behaviors in the Behavioral Intervention Context

Abstract

Despite encouraging evidence for the efficacy of comprehensive and intensive behavioral intervention (CIBI) programs, the majority of studies have focused on relatively narrow, deficit-focused outcomes. More specifically, although adaptive social communication and interaction (SCI) are essential for facilitative functioning, the majority of studies have utilized instruments that capture only the severity of SCI symptoms. Thus, given the importance of the comprehensive and appropriate characterization of distinct SCI adaptive skills in CIBI, in this review, based on PubMed search strategies to identify relevant published articles, we provide a critical appraisal of two of the most commonly used adaptive functioning measures-the Vineland Adaptive Behavior Scales-Third Edition (Vineland-3) and the Adaptive Behavior Assessment System-Third Edition (ABAS-3), for characterizing SCI in the behavioral intervention context. The review focused on periodic outcome and treatment planning assessment in people with autism spectrum disorder receiving CIBI programs. Instrument technical manuals were reviewed and a PubMed search was used to identify published manuscripts, with relevance to Vineland-3 and ABAS-3 development, psychometric properties, or measure interpretation. Instrument analysis begins by introducing the roles of periodic outcome assessment for CIBI programs. Next, the Vineland-3 and ABAS-3 are evaluated in terms of their development processes, psychometric characteristics, and the practical aspects of their implementation. Examination of psychometric evidence for each measure demonstrated that the evidence for several key psychometric characteristics is either unavailable or suggests less-than-desirable properties. Evaluation of practical considerations for implementation revealed weaknesses in ongoing intervention monitoring and clinical decision support. The Vineland-3 and ABAS-3 have significant strengths for cross-sectional outpatient mental health assessment, particularly as related to the identification of intellectual disability, but also substantial weaknesses relevant to their application in CIBI outcome assessment. Alternative approaches are offered, including adopting measures specifically developed for the CIBI context.

Keywords: ABAS-3; Vineland-3; adaptive behavior; autism spectrum disorder; behavioral intervention.

Figure 1

A content review of adaptive function and selected autism symptom measures for total autism symptoms, the SCI domain, and specific SCI dimensions identified in the factor-analytic literature (Chetcuti et al., 2024; Frazier & Hardan, 2017; Phillips et al., 2019; Simmons et al., 2024; Uljarevic et al., 2020a). Note, SRS-2 = Social Responsiveness Scale-Second Edition, SCQ = Social Communication Questionnaire, PDDBI = Pervasive Developmental Disorder Behavior Inventory, RBS-R = Repetitive Behavior Scale-Revised, ASDQ-3 = Autism Symptom Dimensions Questionnaire-Version 3.0.

Figure 2

Domains of coverage for initial and periodic CIBI outcome assessment, derived from prior processes for eliciting input from caregiver/patient and clinician/scientist ASD stakeholders. Several prior large-sample studies support the Social Communication/Interaction framework detailed above (Chetcuti et al., 2024; Frazier & Hardan, 2017; Phillips et al., 2019; Simmons et al., 2024; Uljarevic et al., 2020a). Note, cognition, including general cognitive ability and verbal and nonverbal aspects of intelligence, is a frequent outcome measure in research studies but would be very difficult to include in an initial and periodic CIBI outcome assessment process for everyday practice due to the additional time and expertise required for cognitive test administration and the additional cost added to the process. Unfortunately, as noted above, because both the Vineland-3 and ABAS-3 include a large but not comprehensive list of SCIs, adding skills-based assessments to compensate for incomplete coverage is often not practically feasible. This approach would lead to very long and burdensome assessments without adequately covering all of the relevant domains that are most essential for treatment planning. Importantly, the Vineland-3 and ABAS-3 did not systematically engage with the relevant stakeholders, in particular with autistic individuals, during the initial development phase. In addition to the increased awareness of the need for an inclusive approach to research design, a number of different gold-standard measurement development and validation frameworks, including the PROMIS (Patient-Reported Outcome Measurement Information System) framework, have emphasized the need to engage the relevant stakeholders in order to ensure not only that the measures do not miss any behaviors that are important to capture, but to also facilitate the acceptability of the instrument. Thus, we review three alternative approaches to improving CIBI outcome assessment: combining skills-based assessments with newly developed standardized, norm-referenced assessments; adopting the International Center for Health Outcome Measurement (ICHOM) ASD standard set (Joseph et al., 2024); and developing revised versions of the existing adaptive behavior measures while coupling them with measures of additional domains relevant to CIBI outcome assessment.