Statistical item analysis of the examination in anesthesiology for medical students using the Rasch model

Abstract

Background: Test evaluation in a clinical curriculum is important for medical education. To identify examinee ability and appropriateness of the test content, this study used the Rasch model to analyze an examination in anesthesiology for medical students.

Methods: Fifty items were administered to 119 fifth- and sixth-year medical students in the exam. The Rasch model was used to perform item analysis of the examination. Misfit items or examinees were excluded first, then test reliability was assessed with reliability indices. Both examinee ability measures and item difficulty were estimated and expressed in a common logit unit, which could be further translated into probability of correct responses in the examination.

Results: After the exclusion of two misfit items and one misfit examinee, the estimated test reliability was only 0.63. The mean item difficulty was set at 0 by definition (SD = 2.02) and the mean examinee ability was 1.56 (SD = 0.71), which means that the examinees were able to correctly answer 83% of items on average. There were 21 items with difficulty lower than the least able examinee and two items with difficulty higher than the most able one.

Conclusion: We demonstrated that statistical item analysis with the Rasch model could provide valuable information related to test reliability, item difficulty and examinee ability, which could be applied to further item modification and future test development of clinical curriculums for medical students.