The CARES classroom observation tool: Psychometrics of an observational measure of culturally responsive practices

Abstract

There is growing interest in improving and assessing teachers' use of culturally responsive practices (CRP) in the classroom, yet relatively few research-based approaches exist to address these measurement gaps. This article presents findings on the psychometric properties of a newly developed classroom observation measure of CRP, called the CARES Observational Assessment Tool, where CARES refers to five theorized domains of CRP. We used a sample of 268 30-min video observations across urban classrooms (Grades 4-9) from the Measures of Effective Teaching Project (Bill and Melinda Gates Foundation, 2014). These videos were scored using the CARES to establish initial construct validity of the measure, assess measurement invariance across teacher and student racial composition, and evaluate convergent, divergent, and concurrent validity within a nomological validity network utilizing item analysis and exploratory structural equation modeling. Results confirmed the five-factor structure of the CARES and reduced the original 41-item version of the CARES to a 19-item classroom observation tool (ω = .73; CFI = .977, TLI = .952, RMSEA = .028, SRMR = .030). Configural measurement invariance was unable to be established across student and teacher racial groups, raising important questions about CRP measurement in the context of racially homogeneous versus heterogeneous classrooms. Some evidence of nomological validity emerged with positive correlations between observed CRP and classroom-level student engagement and academic performance. This study addressed an unmet need regarding the assessment of teacher CRP through observational assessment, which in turn will further inform research regarding the contextual factors associated with indicators of CRP.

Keywords: Classroom observations; Culturally responsive; Exploratory structural equation modeling; Item analysis; Measurement; Validity.