Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Nov 20;18(22):12181.
doi: 10.3390/ijerph182212181.

Identifying and Minimizing Errors in the Measurement of Early Childhood Development: Lessons Learned from the Cognitive Testing of the ECDI2030

Claudia Cappa  1 , Nicole Petrowski  1 , Elga Filipa De Castro  1 , Emily Geisen  2 , Patricia LeBaron  3 , Betania Allen-Leigh  4 , Jean Marie Place  5 , Paul J Scanlon  6
Affiliations

Identifying and Minimizing Errors in the Measurement of Early Childhood Development: Lessons Learned from the Cognitive Testing of the ECDI2030

Claudia Cappa et al. Int J Environ Res Public Health. .

Abstract

Challenges in measuring early childhood development (ECD) at scale have been documented, yet little is known about the specific difficulties related to questionnaire design and question interpretation. The purpose of this paper is to discuss the challenges of measuring ECD at scale in the context of household surveys and to show how to overcome them. The paper uses examples from the cognitive interviewing exercises that were conducted as part of the methodological work to develop a measure of ECD outcomes, the ECDI2030. It describes the methodological work carried out to inform the selection and improvement of question items and survey implementation tools as a fundamental step to reduce and mitigate systematic measurement error and improve data quality. The project consisted of a total of five rounds of testing, comprising 191 one-on-one, in-depth cognitive interviews across six countries (Bulgaria, India, Jamaica, Mexico, Uganda, and the USA). Qualitative data analysis methods were used to determine matches and mismatches between intention of items and false positives or false negative answers among subgroups of respondents. Key themes emerged that could potentially lead to systematic measurement error in population-based surveys on ECD: (1) willingness of child to perform task versus ability of child to perform task; (2) performing task versus performing task correctly; (3) identifying letters or numbers versus recognizing letters or numbers; (4) consistently performing task versus correctly performing task; (5) applicability of skills being asked versus observability of skills being asked; and (6) language production versus language comprehension. Through an iterative process of testing and subsequent revision, improvements were made to item wording, response options, and interviewer training instructions. Given the difficulties inherent in population-level data collection in the context of global monitoring, this study's findings confirm the importance of cognitive testing as a crucial step in careful, culturally relevant, and sensitive questionnaire design and as a means to reduce response bias in cross-cultural contexts.

Keywords: cognitive testing; cultural relevance; early childhood; questionnaire design.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

References

    1. Edward A., Frongillo E.A., Baranowski T., Subar A.F., Tooze J.A., Kirkpatrick S.I. Establishing Validity and Cross-context Equivalence of Measures and Indicators. J. Acad. Nutr. Diet. 2019;119:1817–1830. doi: 10.1016/j.jand.2018.09.005. - DOI - PubMed
    1. Colbert C.Y., French J.C., Arroliga A.C., Bierer S.B. Best Practice Versus Actual Practice: An Audit of Survey Pretesting Practices Reported in a Sample of Medical Education Journals. Med. Educ. Online. 2019;24 doi: 10.1080/10872981.2019.1673596. - DOI - PMC - PubMed
    1. Bennett C., Khangura S., Brehaut J.C., Graham I.D., Moher D., Potter B.K., Grimshaw J.M. Reporting Guidelines for Survey Research: An Analysis of Published Guidance and Reporting Practices. PLoS Med. 2011;8:e1001069. doi: 10.1371/journal.pmed.1001069. - DOI - PMC - PubMed
    1. McCoy D.C., Peet E.D., Ezzati M., Danaei G., Black M.M., Sudfeld C.R., Fawzi W., Fink G. Early Childhood Developmental Status in Low and Middle-Income Countries: National, Regional, and Global Prevalence Estimates Using Predictive Modeling. PLoS Med. 2016;13:e1002034. doi: 10.1371/journal.pmed.1002034. - DOI - PMC - PubMed
    1. Fernald L., Prado E., Kariger P., Raikes A. A Toolkit for Measuring Early Childhood Development in Low- and Middle-Income Countries. The World Bank; Washington, DC, USA: 2017.

LinkOut - more resources