A Systematic Review and Meta-Analysis of a Measure of Staff/Child Interaction Quality (the Classroom Assessment Scoring System) in Early Childhood Education and Care Settings and Child Outcomes

Abstract

The quality of staff/child interactions as measured by the Classroom Assessment Scoring System (CLASS) in Early Childhood Education and Care (ECEC) programs is thought to be important for children's outcomes. The CLASS is made of three domains that assess Emotional Support, Classroom Organization and Instructional Support. It is a relatively new measure that is being used increasingly for research, quality monitoring/accountability and other applied purposes. Our objective was to evaluate the association between the CLASS and child outcomes. Searches of Medline, PsycINFO, ERIC, websites of large datasets and reference sections of all retrieved articles were conducted up to July 3, 2015. Studies that measured association between the CLASS and child outcomes for preschool-aged children who attended ECEC programs were included after screening by two independent reviewers. Searches and data extraction were conducted by two independent reviewers. Thirty-five studies were systematically reviewed of which 19 provided data for meta-analyses. Most studies had moderate to high risk of bias. Of the 14 meta-analyses we conducted, associations between Classroom Organization and Pencil Tapping and between Instructional Support and SSRS Social Skills were significant with pooled correlations of .06 and .09 respectively. All associations were in the expected direction. In the systematic review, significant correlations were reported mainly from one large dataset. Substantial heterogeneity in use of the CLASS, its dimensions, child outcomes and statistical measures was identified. Greater consistency in study methodology is urgently needed. Given the multitude of factors that impact child development it is encouraging that our analyses revealed some, although small, associations between the CLASS and children's outcomes.

Fig 1. Flow diagram for study selection.

Adapted from Moher, 2009 [71].

Fig 2. Systematic review of associations between the CLASS Total Score or Dimension and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. Total Score (Two Factor Solution) = Total Score for Emotional Climate and Instructional Climate; Total Score (9 Scale Version) = Total Score with Language Modeling and Literacy Focus dimension not included. For more details, see Table 2 in this manuscript. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. ^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).^fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].^ANational Center for Early Development and Learning Dataset (NCEDL); ^BHead Start Family and Children Experiences Survey (FACES 2006 Cohort); ^MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort); ^UPreschool Curriculum Evaluation Research (PCER, 1999–2003);^XOhio Virginia (2005–2006, 2006–2007).

Fig 3. Systematic review of associations between the CLASS domain of Classroom Organization and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson's Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File.^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).^fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].^ANational Center for Early Development and Learning Dataset (NCEDL); ^MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).

Fig 4. Systematic review of associations between the CLASS domain of Emotional Support and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. ^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). ^fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].^ANational Center for Early Development and Learning Dataset (NCEDL); ^MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).

Fig 5. Systematic review of associations between the CLASS domain of Instructional Support and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. ^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).^fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].^ANational Center for Early Development and Learning Dataset (NCEDL); ^BHead Start Family and Children Experiences Survey (FACES 2006 Cohort); ^MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).

Fig 6. Systematic review of associations between the CLASS domain of Emotional Climate and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. ^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). ^ANational Center for Early Development and Learning Dataset (NCEDL).

Fig 7. Systematic review of associations between the CLASS domain of Instructional Climate and child outcomes.

^a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. ^aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. ^bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. ^cSamples within papers are described in more detail in Table 2 in this manuscript. ^dAcronyms for child outcomes are listed in S5 File. ^eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). ^ANational Center for Early Development and Learning Dataset (NCEDL).

Fig 8. Meta-analyses of associations between the CLASS domain of Classroom Organization and child outcomes.

Fig 9. Meta-analyses of associations between the CLASS domain of Emotional Support and child outcomes.

Fig 10. Meta-analyses of associations between the CLASS domain of Instructional Support and child outcomes.