Agreement and Reliability of Parental Reports and Direct Screening of Developmental Outcomes in Toddlers at Risk

Abstract

Developmental screening is a practice that directly benefits vulnerable and low-income families and children when it is regular and frequently applied. A developmental screening tool administered by parents called CARE is tested. CARE contains a compilation of activities to report and enhance development at home. Hundred and fifty-seven families in Bogotá (Colombia) initially responded to a call to participate in developmental screening tools' validation and reliability study. All children (Average: 42.7 months old; SD: 9.4; Min: 24, Max: 58) were screened directly by trained applicants using a Spanish version of the Denver Developmental Screening test [i.e., the Haizea-Llevant (HLL) screening table]. After a first screening, 61 dyads were positive for follow-up and received a second HLL screening. Fifty-two out of 61 dyads use and returned CARE booklet after 1-month screening at home. The comparative analysis for parent reports using CARE and direct screening observation included (a) the effects of demographic variables on overall and agreement, (b) agreement and congruence between the CARE report classification and direct screening classification ("At risk" or "Not at risk"), (c) receiver operating characteristic analysis, (d) item-Level agreement for specific developmental domains, and (e) acceptability and feasibility analysis. Results and conclusions show the parental report using the CARE booklet as a reliable screening tool that has the potential to activate alerts for an early cognitive delay that reassure clinicians and families to further specialized and controlled developmental evaluations and act as a screen for the presence of such delay in four developmental dimensions.

Keywords: children at risk; developmental screening; low-middle income countries; parental reports; receiver operating characteristic (ROC) analysis; reliability and agreement studies.

FIGURE 1

Consort diagram for participants called for screened with the Haizea-Llevant screening table and to use CARE at home. HLL, Haizea-Llevant; CARE, The Compilation of Activities to Report and Enhance development booklet. One-month pass between the positive Follow-up and the caregivers return of CARE booklet used as screening tool.

FIGURE 2

(A,B) Examples of Caution and Delay answers in “Identify colors” item in Haizea-Llevant.

FIGURE 3

Report icons of parent–child interaction in CARE booklet.

FIGURE 4

Fagan’s nomogram showing probability of children At risk after parents report using CARE booklet. Probabilities were calculated based on the screening with Haizea-Llevant table (HLL). Positive At risk diagnosis (blue arrow) refers to typical or non-specific appearance, and Not at risk diagnosis (red arrow) to atypical or negative appearance in CARE. Precision is given as 95% confidence interval. Risk prevalence is derived from the number of At risk positive and Not at risk participants after screening with HLL. LR+, positive likelihood ratio; LR–, negative likelihood ratio; NPV, negative predictive value; PPV, positive predictive value. Diagnostic test calculator (version 2010042101). Copyright (c) 2002-2006 by Alan Schwartz < alansz@uic.edu >.

FIGURE 5

Receiver operating characteristic (ROC) binormal curve for CARE and Haizea-Llevant classification for the total sample (n = 52). This ROC curves plot use web-based calculator for ROC curves (http://www.jrocfit.org). Gray lines indicate 95% confidence interval of the fitted ROC curve. ROC analysis plot for each possible cut-off points of the relevant CARE scale, the true-positive proportion (sensitivity = 95%) against the false-positive proportion (1– specificity). A perfect test would have an area under the curve (AUC) of 1 and the curve would pass through the upper left corner of the plot (100% sensitivity, 100% specificity). In this study, Trapezoidal (Wilcoxon) area/AUC = 0.89 (SE = 0.04) and the Area under the fitted curve (Az) = 0.90 (SE = 0.052).