DIF Analysis with Unknown Groups and Anchor Items

Abstract

Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption - the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an $L_1$ -regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test.

Keywords: differential item functioning; lasso; latent DIF; latent class analysis; measurement invariance.

Fig. 1

Path diagram of the proposed model, where the dashed lines indicate the DIF effects.

Algorithm 1

Regularised estimation and model selection.

Algorithm 2

An EM algorithm for solving (3).

Fig. 2

RMSE for $J=25$ under the 2-group setting.

Fig. 3

RMSE for $J=50$ under the 2-group setting.

Fig. 4

The RMSEs for under the 3-group setting.

Algorithm 3

Line Search Algorithm

Algorithm 4

Soft Threshold Function

Algorithm 5

Proximal Gradient Function