Improving the assessment of measurement invariance: Using regularization to select anchor items and identify differential item functioning

Abstract

A common challenge in the behavioral sciences is evaluating measurement invariance, or whether the measurement properties of a scale are consistent for individuals from different groups. Measurement invariance fails when differential item functioning (DIF) exists, that is, when item responses relate to the latent variable differently across groups. To identify DIF in a scale, many data-driven procedures iteratively test for DIF one item at a time while assuming other items have no DIF. The DIF-free items are used to anchor the scale of the latent variable across groups, identifying the model. A major drawback to these iterative testing procedures is that they can fail to select the correct anchor items and identify true DIF, particularly when DIF is present in many items. We propose an alternative method for selecting anchors and identifying DIF. Namely, we use regularization, a machine learning technique that imposes a penalty function during estimation to remove parameters that have little impact on the fit of the model. We focus specifically here on a lasso penalty for group differences in the item parameters within the two-parameter logistic item response theory model. We compare lasso regularization with the more commonly used likelihood ratio test method in a 2-group DIF analysis. Simulation and empirical results show that when large amounts of DIF are present and sample sizes are large, lasso regularization has far better control of Type I error than the likelihood ratio test method with little decrement in power. This provides strong evidence that lasso regularization is a promising alternative for testing DIF and selecting anchors. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Figure 1.. False positives for Reg-DIF and IRT-LR-DIF (empirical Type I error)

Figure 2.. True positives for Reg-DIF and IRT-LR-DIF (empirical power)

Figure 3.. Mean squared error for DIF estimates that were not present in the population

Note. MSE for sample sizes of 1000, collapsing across number of scale items (i.e., 6 and 12). DIF estimates were simulated to have DIF = 0 in the population.

Figure 4.. Mean squared error for DIF estimates that were present in the population

Note. MSE for sample sizes of 1000, collapsing across number of scale items (i.e., 6 and 12). DIF estimates were simulated to have DIF ≠ 0 in the population.