Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 1:01466216251351949.
doi: 10.1177/01466216251351949. Online ahead of print.

Detecting DIF with the Multi-Unidimensional Pairwise Preference Model: Lord's Chi-square and IPR-NCDIF Methods

Lavanya S Kumar  1 , Naidan Tu  2 , Sean Joo  3 , Stephen Stark  1
Affiliations

Detecting DIF with the Multi-Unidimensional Pairwise Preference Model: Lord's Chi-square and IPR-NCDIF Methods

Lavanya S Kumar et al. Appl Psychol Meas. .

Abstract

Multidimensional forced choice (MFC) measures are gaining prominence in noncognitive assessment. Yet there has been little research on detecting differential item functioning (DIF) with models for forced choice measures. This research extended two well-known DIF detection methods to MFC measures. Specifically, the performance of Lord's chi-square and item parameter replication (IPR) methods with MFC tests based on the Multi-Unidimensional Pairwise Preference (MUPP) model was investigated. The Type I error rate and power of the DIF detection methods were examined in a Monte Carlo simulation that manipulated sample size, impact, DIF source, and DIF magnitude. Both methods showed consistent power and were found to control Type I error well across study conditions, indicating that established approaches to DIF detection work well with the MUPP model. Lord's chi-square outperformed the IPR method when DIF source was statement discrimination while the opposite was true when DIF source was statement threshold. Also, both methods performed similarly and showed better power when DIF source was statement location, in line with previous research. Study implications and practical recommendations for DIF detection with MFC tests, as well as limitations, are discussed.

Keywords: differential item functioning; item response theory; linking; measurement invariance; multi-unidimensional pairwise preference model; multidimensional forced choice.

PubMed Disclaimer

Conflict of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.
Figure 1.
Comparison of Reference and Focal MUPP IRFs of a Simulated Item in 1.20 𝛼 DIF Condition. (a) Reference group IRF (𝛼𝑠= 0.98, δ𝑠= 0.79, τ𝑠= –0.70, 𝛼𝑡= 1.21, δ𝑡= 1.10, τ𝑡= –0.83). (b) Focal group IRF (𝛼𝑠= 2.18). (c) Squared differences between reference and focal group IRFs.
Figure 2.
Figure 2.
Comparison of Reference and Focal MUPP IRFs of a Simulated Item in 1.20 δ DIF Condition. (a) Focal group IRF (δ𝑠= 1.99; Reference group IRF & item parameters in Figure 1(a)). (b) Squared differences between reference and focal group IRFs.
Figure 3.
Figure 3.
Comparison of Reference and Focal MUPP IRFs of a Simulated Item in 1.20 τ DIF Condition. (a) Focal group IRF (τ𝑠= –1.90; Reference group IRF & item parameters in Figure 1(a)). (b) Squared differences between reference and focal group IRFs.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Barrick M. R., Mount M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44(1), 1–26. 10.1111/j.1744-6570.1991.tb00688.x - DOI
    1. Birnbaum A. (1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord F. M., Novick M. R. (Eds.), Statistical theories of mental test scores (pp. 397–472). Addison-Wesley.
    1. Brooks S. P., Gelman A. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455. 10.1080/10618600.1998.10474787 - DOI
    1. Brown A., Maydeu-Olivares A. (2011). Item response modeling of forced choice questionnaires. Educational and Psychological Measurement, 71(3), 460–502. 10.1177/0013164410375112 - DOI
    1. Candell G. L., Drasgow F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12(3), 253–260. 10.1177/014662168801200304 - DOI

LinkOut - more resources