Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Sep 20;4(1):e12198.
doi: 10.1002/jcv2.12198. eCollection 2024 Mar.

Measurement bias in caregiver-report of early childhood behavior problems across demographic factors in an ECHO-wide diverse sample

Shuting Zheng  1 , Maxwell Mansolf  2 , Monica McGrath  3 , Marie L Churchill  3 , Traci A Bekelman  4 , Patricia A Brennan  5 , Amy E Margolis  6 , Sara S Nozadi  7 , Theresa M Bastain  8 , Amy J Elliott  9 , Kaja Z LeWinn  1 , Julie A Hofheimer  10 , Leslie D Leve  11 , Brandon Rennie  12 , Emily Zimmerman  13 , Carmen A Marable  14 , Cindy T McEvoy  15 , Chang Liu  16 , Alexis Sullivan  17 , Tracey J Woodruff  1 , Samiran Ghosh  18 , Bennett Leventhal  19 , Assiamira Ferrara  20 , Johnnye Lewis  21 , Somer Bishop  22 ; Environmental influences on Child Health Outcomes
Affiliations

Measurement bias in caregiver-report of early childhood behavior problems across demographic factors in an ECHO-wide diverse sample

Shuting Zheng et al. JCPP Adv. .

Abstract

Background: Research and clinical practice rely heavily on caregiver-report measures, such as the Child Behavior Checklist 1.5-5 (CBCL/1.5-5), to gather information about early childhood behavior problems and to screen for child psychopathology. While studies have shown that demographic variables influence caregiver ratings of behavior problems, the extent to which the CBCL/1.5-5 functions equivalently at the item level across diverse samples is unknown.

Methods: Item-level data of CBCL/1.5-5 from a large sample of young children (N = 9087) were drawn from 26 cohorts in the Environmental influences on Child Health Outcomes program. Factor analyses and the alignment method were applied to examine measurement invariance (MI) and differential item functioning (DIF) across child (age, sex, bilingual status, and neurodevelopmental disorders), and caregiver (sex, education level, household income level, depression, and language version administered) characteristics. Child race was examined in sensitivity analyses.

Results: Items with the most impactful DIF across child and caregiver groupings were identified for Internalizing, Externalizing, and Total Problems. The robust item sets, excluding the high DIF items, showed good reliability and high correlation with the original Internalizing and Total Problems scales, with lower reliability for Externalizing. Language version of CBCL administration, education level and sex of the caregiver respondent showed the most significant impact on MI, followed by child age. Sensitivity analyses revealed that child race has a unique impact on DIF over and above socioeconomic status.

Conclusions: The CBCL/1.5-5, a caregiver-report measure of early childhood behavior problems, showed bias across demographic groups. Robust item sets with less DIF can measure Internalizing and Total Problems equally as well as the full item sets, with slightly lower reliability for Externalizing, and can be crosswalked to the metric of the full item set, enabling calculation of normed T scores based on more robust item sets.

Keywords: behavior problems; behavioral measures; pre‐school children; psychometrics.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that they have no competing or potential conflicts of interest.

Figures

FIGURE 1
FIGURE 1
Model fit distributions for factor analysis models across three latent constructs. SRMSR, Standardized Root Mean Squared Residual; RMSEA, Root Mean Squared Error of Approximation.
FIGURE 2
FIGURE 2
Items with Unsigned Item Difference in the Sample (UIDS) > 0.1 in Univariate Groupings across Latent Constructs. Ext, Externalizing domain; Ext + Tot, Externlizing and Total Problems; Int, Internalizing domain; Int + Tot, Internalizing and Total Problems; Tot, Total Problems; UIDS, Unsigned Item Difference in the Sample; RespSex, Respondent Sex; NDD, Neurodevelopmental Disorder. Items with no significant differential item functioning (DIF) or no UIDS >0.1 are excluded from the figure. Items are grouped by domains and sorted in decreasing order of the number of groupings with UIDS >0.1 of items; groupings are sorted by the number of items with UIDS >0.1 within the grouping. The numbers on the two right‐hand columns show the number of groupings with UIDS >0.1 for the specific items within domains (Internalizing or Externalizing, and Total Problems).
FIGURE 3
FIGURE 3
Item response theory (IRT) Reliability for Full and Robust Item Sets. Full, full CBCL item set; Robust, item set with high‐DIF items removed.
FIGURE 4
FIGURE 4
Distribution of Unsigned Item Difference in the Sample (UIDS) by Grouping and Latent Constructs. NDD, Any Neurodevelopment Disorders; CareDep, Caregiver Depression with a cutoff of T‐score ≥ 60 for clinical range; RespSex, Respondent Sex.
FIGURE 5
FIGURE 5
Signed Test Difference in the Sample (STDS) for Univariate Groupings. RespSex, Respondent Sex; RespFemale, Female Respondent; RespMale, Male Respondent; CareDep, Caregiver Depression with clinical threshold of T = 60; HighDep, Depression T score greater than or equal to 60; LowDep, Depression T score less than 60; NDD, Neurodevelopmental Disorder.
See this image and copyright information in PMC

References

    1. Achenbach, T. M. , Ivanova, M. Y. , Rescorla, L. A. , Turner, L. V. , & Althoff, R. R. (2016). Internalizing/Externalizing problems: Review and recommendations for clinical and research applications. Journal of the American Academy of Child & Adolescent Psychiatry, 55(8), 647–656. 10.1016/j.jaac.2016.05.012 - DOI - PubMed
    1. Achenbach, T. M. , & Rescorla, L. A. (2001). Child behavior checklist 1.5 to 5. University of Vermont, Research Center for Children, Youth, & Families.
    1. Albano, A. D. (2016). equate: An R package for observed‐score linking and equating. Journal of Statistical Software, 74(8), 1–36. 10.18637/jss.v074.i08 - DOI
    1. Bayer, J. K. , Ukoumunne, O. C. , Mathers, M. , Wake, M. , Abdi, N. , & Hiscock, H. (2012). Development of children’s internalising and externalising problems from infancy to five years of age. Australian and New Zealand Journal of Psychiatry, 46(7), 659–668. SAGE Publications Ltd. 10.1177/0004867412450076 - DOI - PubMed
    1. Blackwell, C. K. , Tang, X. , Elliott, A. J. , Thomes, T. , Louwagie, H. , Gershon, R. , Schalet, B. D. , & Cella, D. (2021). (Vol. 33, pp. 610–618). American Psychological Association. 10.1037/pas0001009.Developing a common metric for depression across adulthood: Linking PROMIS depression with the Edinburgh Postnatal Depression Scale Psychological Assessment 7 - DOI - PMC - PubMed