Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Dec 5:14:1288177.
doi: 10.3389/fpsyg.2023.1288177. eCollection 2023.

Contrasting multistage and computer-based testing: score accuracy and aberrant responding

Georgios Sideridis  1   2 , Hanan Ghamdi  3 , Omar Zamil  3
Affiliations

Contrasting multistage and computer-based testing: score accuracy and aberrant responding

Georgios Sideridis et al. Front Psychol. .

Abstract

The goal of the present study was to compare and contrast the efficacy of a multistage testing (MST) design using three paths compared to a traditional computer-based testing (CBT) approach involving items across all ability levels. Participants were n = 627 individuals who were subjected to both a computer-based testing (CBT) instrument and a measure constructed using multistage testing to route individuals of low, middle, and high ability to content that was respective to their ability level. Comparisons between the medium of testing involved person ability accuracy estimates and evaluation of aberrant responding. The results indicated that MST assessments deviated markedly from CBT assessments, especially for low- and high-ability individuals. Test score accuracy was higher overall in MST compared to CBT, although error of measurement was enhanced for high-ability individuals during MST compared to CBT. Evaluating response patterns indicated significant amounts of Guttman-related errors during CBT compared to MST using person-fit aberrant response indicators. It was concluded that MST is associated with significant benefits compared to CBT.

Keywords: aberrant responding; carelessness; guessing; item response theory; multistage testing; person fit statistics.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Scatterplot displaying the relationship between MST (vertical axis) and CBT (horizontal axis) factor scores. EE, Easy–Easy track, MM, Medium–Medium track, DD, Difficult–Difficult track.
Figure 2
Figure 2
Theta estimates by track and condition (upper panel) and standard errors (lower panel).
Figure 3
Figure 3
Person response functions (PRFs) for participant no. 12 when evaluated using MST (left) and CBT (right) measurements.
See this image and copyright information in PMC

References

    1. Abramson L. Y., Metalsky G. I., Alloy L. B. (1989). Hopelessness depression: a theory-based subtype of depression. Psychol. Rev. 96, 358–372. doi: 10.1037/0033-295X.96.2.358 - DOI
    1. Beck M. F., Albano A. D., Smith W. M. (2019). Person-fit as an index of inattentive responding: a comparison of methods using polytomous survey data. Appl. Psychol. Meas. 43, 374–387. doi: 10.1177/0146621618798666, PMID: - DOI - PMC - PubMed
    1. Cohen J. (1992). A power primer. Psychol. Bull. 112, 155–159. doi: 10.1037/0033-2909.112.1.155 - DOI - PubMed
    1. Dimitrov D. M. (2018). The delta-scoring method of tests with binary items: a note on true score estimation and equating. Educ. Psychol. Meas. 78, 805–825. doi: 10.1177/0013164417724187, PMID: - DOI - PMC - PubMed
    1. Emons W. H., Sijtsma K., Meijer R. R. (2005). Global, local, and graphical person-fit analysis using person-response functions. Psychol. Methods 10, 101–119. doi: 10.1037/1082-989X.10.1.101, PMID: - DOI - PubMed

LinkOut - more resources