Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Mar;31(1):1-9.
doi: 10.3946/kjme.2019.113. Epub 2019 Mar 1.

The relationship between classical item characteristics and item response time on computer-based testing

Yoo-Mi Chae  1 , Seok Gun Park  1   2 , Ilyong Park  3
Affiliations

The relationship between classical item characteristics and item response time on computer-based testing

Yoo-Mi Chae et al. Korean J Med Educ. 2019 Mar.

Abstract

Purpose: This study investigated the relationship between the item response time (iRT) and classic item analysis indicators obtained from computer-based test (CBT) results and deduce students' problem-solving behavior using the relationship.

Methods: We retrospectively analyzed the results of the Comprehensive Basic Medical Sciences Examination conducted for 5 years by a CBT system in Dankook University College of Medicine. iRT is defined as the time spent to answer the question. The discrimination index and the difficulty level were used to analyze the items using classical test theory (CTT). The relationship of iRT and the CTT were investigated using a correlation analysis. An analysis of variance was performed to identify the difference between iRT and difficulty level. A regression analysis was conducted to examine the effect of the difficulty index and discrimination index on iRT.

Results: iRT increases with increasing difficulty index, and iRT tends to decrease with increasing discrimination index. The students' effort is increased when they solve difficult items but reduced when they are confronted with items with a high discrimination. The students' test effort represented by iRT was properly maintained when the items have a 'desirable' difficulty and a 'good' discrimination.

Conclusion: The results of our study show that an adequate degree of item difficulty and discrimination is required to increase students' motivation. It might be inferred that with the combination of CTT and iRT, we can gain insights about the quality of the examination and test behaviors of the students, which can provide us with more powerful tools to improve them.

Keywords: Computer-based test; Difficulty index; Discrimination index; Item response time.

PubMed Disclaimer

Conflict of interest statement

Conflicts of interest

No potential conflict of interest relevant to this article was reported. The authors alone are responsible for the content and writing of the paper.

Figures

Fig. 1.
Fig. 1.. Basic Configuration of the Web-Based Computer-Based Test System for This Study
Fig. 2.
Fig. 2.. Scatter Plot of the Difficulty Index for Item Response Time
Fig. 3.
Fig. 3.. Scatter Plot of the Discrimination Index for Item Response Time
Fig. 4.
Fig. 4.. Three-dimensional Scatter Plot of the Discrimination Index, Difficulty Index, and Item Response Time
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Son CG. Curriculum & educational evaluation. Seoul, Korea: Teayong; 2012.
    1. Lim HS, Lee YM, Ahn DS, Lee JY, Im H. Item analysis of clinical performance examination using item response theory and classical test theory. Korean J Med Educ. 2007;19(3):185–195.
    1. University of Washington Understanding item analyses. http://www.washington.edu/assessment/scanning-scoring/scoring/reports/it.... Accessed July 3, 2018.
    1. Huh S. Preparing the implementation of computerized adaptive testing for high-stakes examinations. J Educ Eval Health Prof. 2008;5:1. - PMC - PubMed
    1. Wise SL. An investigation of the differential effort received by items on a low-stakes computer-based test. Appl Meas Educ. 2006;19(2):95–114.

LinkOut - more resources