The Alcohol Intervention Mechanisms Scale (AIMS): Preliminary Reliability and Validity of a Common Factor Observational Rating Measure

Abstract

The present work provides an overview, and pilot reliability and validity for the Alcohol Intervention Mechanisms Scale (AIMS). The AIMS measures therapist interventions that occur broadly across modalities of behavioral treatment for alcohol use disorder. It was developed based on identified commonalities in the function rather than content of therapist interventions in observed therapy sessions, as well as from existing observer rating systems. In the AIMS, the primary function areas are: explore (four behavior count codes), teach (five behavior count codes), and connect (three behavior count codes). Therapist behavior counts provide a frequency rating of occurrence (i.e., adherence). The three functions (explore, teach, connect) are then rated on global skillfulness, which provides a quality valence (i.e., competence) to the entire session. In the present study, three independent raters received roughly 30 hours of training on the use of the AIMS by the first author. Data were a sample of therapy session audio files from a Project MATCH clinical research site. Reliability results showed generally good performance for the measure. Specifically, 2-way mixed intraclass coefficients were 'excellent', ranging from .94 to .99 for function summary scores, while prevalence-adjusted, bias-adjusted kappa for global skillfulness measures were in the 'fair' to 'moderate' range (k=.36 to.40). Internal consistency reliability was acceptable, as were preliminary factor models by behavioral treatment function (i.e., explore, teach, connect). However, confirmatory fit for the subsequent three factor model was poor. In concurrent validity analyses, AIMS summary and skillfulness scores showed associations with relevant Project MATCH criterion measures (i.e., MATCH Tape Rating Scale) that were consistent with expectations. The AIMS is a promising and reliable observational measure of three proposed common functions of behavioral alcohol treatment.

Keywords: Alcohol; Common factors; Process research; Project MATCH; Psychometrics.