Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jun;84(3):481-509.
doi: 10.1177/00131644231180529. Epub 2023 Jun 26.

An Evaluation of Fit Indices Used in Model Selection of Dichotomous Mixture IRT Models

Sedat Sen  1 , Allan S Cohen  2
Affiliations

An Evaluation of Fit Indices Used in Model Selection of Dichotomous Mixture IRT Models

Sedat Sen et al. Educ Psychol Meas. 2024 Jun.

Abstract

A Monte Carlo simulation study was conducted to compare fit indices used for detecting the correct latent class in three dichotomous mixture item response theory (IRT) models. Ten indices were considered: Akaike's information criterion (AIC), the corrected AIC (AICc), Bayesian information criterion (BIC), consistent AIC (CAIC), Draper's information criterion (DIC), sample size adjusted BIC (SABIC), relative entropy, the integrated classification likelihood criterion (ICL-BIC), the adjusted Lo-Mendell-Rubin (LMR), and Vuong-Lo-Mendell-Rubin (VLMR). The accuracy of the fit indices was assessed for correct detection of the number of latent classes for different simulation conditions including sample size (2,500 and 5,000), test length (15, 30, and 45), mixture proportions (equal and unequal), number of latent classes (2, 3, and 4), and latent class separation (no-separation and small separation). Simulation study results indicated that as the number of examinees or number of items increased, correct identification rates also increased for most of the indices. Correct identification rates by the different fit indices, however, decreased as the number of estimated latent classes or parameters (i.e., model complexity) increased. Results were good for BIC, CAIC, DIC, SABIC, ICL-BIC, LMR, and VLMR, and the relative entropy index tended to select correct models most of the time. Consistent with previous studies, AIC and AICc showed poor performance. Most of these indices had limited utility for three-class and four-class mixture 3PL model conditions.

Keywords: fit indices; maximum likelihood estimation; mixture IRT models; model selection.

PubMed Disclaimer

Conflict of interest statement

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1
Figure 1
Correct Model Selection Frequency for Mixture Rasch Model
Figure 2
Figure 2
Correct Model Selection Frequency for Mixture 2PL Model Note. 2PL = two-parameter logistic.
Figure 3
Figure 3
Correct Model Selection Frequency for Mixture 3PL Model Note. 3PL = three-parameter logistic.
See this image and copyright information in PMC

References

    1. Akaike H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.
    1. Alexeev N., Templin J., Cohen A. S. (2011). Spurious latent classes in the mixture Rasch model. Journal of Educational Measurement, 48(3), 313–332.
    1. Al Hakmani R. (2018). Bayesian estimation of mixture IRT models using NUTS [Unpublished doctoral dissertation]. Southern Illinois University at Carbondale.
    1. Al Hakmani R., Sheng Y. (2023). Empirical evaluation of fully Bayesian information criteria for mixture IRT models using NUTS. Behaviormetrika, 50, 93–120.
    1. Aryadoust V. (2015). Fitting a mixture Rasch model to English as a foreign language listening tests: The role of cognitive and background variables in explaining latent differential item functioning. International Journal of Testing, 15(3), 216–238.

LinkOut - more resources