EuleApp©: a computerized adaptive assessment tool for early literacy skills

Abstract

Introduction: Ample evidence indicates that assessing children's early literacy skills is crucial for later academic success. This assessment enables the provision of necessary support and materials while engaging them in the culture of print and books before school entry. However, relatively few assessment tools are available to identify early literacy skills, such as concepts of print, print awareness, phonological awareness, word awareness, alphabet knowledge, and early reading. The digital landscape presents new opportunities to enhance these assessments and provide enriching early literacy experiences. This study examines the psychometric properties of an adaptive assessment tool, EuLeApp©, focusing on its reliability and concurrent validity.

Methods: Data involved 307 German kindergarten children (M_age = 64 months old, range = 45-91). A Computerized Adaptive Testing (CAT) method, grounded in Item Response Theory (IRT), was employed to develop an adaptive digital tool for assessing early literacy competencies. We utilized an automatic item selection procedure based on item difficulty and discrimination parameters for the 183-item pool to ensure a precise and efficient assessment tailored to each child's ability level.

Results: The 4-parameter Logistic (4PL) model was identified as the best-fitting model for adaptive assessment, providing the highest precision in estimating children's abilities within this framework.

Discussions: The findings support the idea that the adaptive digital-based assessment tool EuLeApp© can be used to assess early literacy skills. It also provides a foundation for offering individualized and adaptable learning opportunities embedded in daily routines in daycare centers.

Keywords: computerized adaptive test; digital assessment; early literacy; item response theory; preschool age; psychometric validation.

Figure 1

EuLeApp© subtests with sample test items.

Figure 2

The development of an adaptive app-based early literacy assessment tool.

Figure 3

A typical item characteristic curve for the 4PL IRT model. P(θ) represents the probability of a correct response given the ability level. ${\chi }_j$ (ability). This represents the ability level of the individual, typically ranging from very low to very high values. ${\Upsilon }_j$ (probability of correct response). This represents the probability of answering the item correctly, ranging from 0 to 1. 4PL = four-parameter logistic.

Figure 4

Item difficulties in the calibrated item bank. MDISC: Item Discrimination Index, MDIFF: Item Difficulty Index. The MDIFF values were multiplied by −1 so that the difficulty of the items increases from left to right in the diagram.

Figure 5

Distribution of item selection per test based on stop rule. The figure shows the distribution of the number of items completed (x-axis labeled “number of items”) and their frequency (y-axis labeled “frequency”).

Figure 6

Test of t-scores across six scales using stop rule when standard errors (SE) < 0.0005. The solid lines represent the average t-scores, while the shaded areas indicate the standard error ranges.