A Mixture IRTree Model for Performance Decline and Nonignorable Missing Data

Abstract

In educational assessments and achievement tests, test developers and administrators commonly assume that test-takers attempt all test items with full effort and leave no blank responses with unplanned missing values. However, aberrant response behavior-such as performance decline, dropping out beyond a certain point, and skipping certain items over the course of the test-is inevitable, especially for low-stakes assessments and speeded tests due to low motivation and time limits, respectively. In this study, test-takers are classified as normal or aberrant using a mixture item response theory (IRT) modeling approach, and aberrant response behavior is described and modeled using item response trees (IRTrees). Simulations are conducted to evaluate the efficiency and quality of the new class of mixture IRTree model using WinBUGS with Bayesian estimation. The results show that the parameter recovery is satisfactory for the proposed mixture IRTree model and that treating missing values as ignorable or incorrect and ignoring possible performance decline results in biased estimation. Finally, the applicability of the new model is illustrated by means of an empirical example based on the Program for International Student Assessment.

Keywords: IRTree; item response theory (IRT); missing not at random; mixture models; performance decline.

Figure 1.

Graphical representation of the mixture IRTree-based model for performance decline and nonignorable missing data. Note. The circles and squares represent the internal nodes and observed responses, respectively. IRTree = item response tree.