How can the examination failure rate be stabilised?

Abstract

Background: The study programme in medicine at the Norwegian University of Science and Technology (NTNU) holds written examinations once annually. The limit to achieving a pass grade is at least 65 % correct answers. The failure rate varies from one year to the next. Our hypothesis was that the variations in the failure rate were caused by a varying degree of difficulty in the examination questions. We investigated whether relative standard-setting methods would reduce the variation in the failure rate without lowering the average limit for a pass grade.

Material and method: Cohen’s relative standard-setting methods correct for the degree of difficulty in the examination questions. They are easy to apply and provide an alternative to setting an absolute limit of 65 % for a pass grade. We used data from 34 examinations for medical studies at the Norwegian University of Science and Technology (NTNU) from the period 2010–2015 and compared the failure rates estimated using the existing assessment method with those produced by Cohen’s methods.

Results: Using the existing 65 % limit for a pass grade, the failure rate varied from 0 % to 13.7 %, with a falling rate at later stages of the studies. With the exception of the examination held in the first year of study, the failure rate was lower and there was less variation in the failure rate with the original as well as the modified Cohen method when compared to the existing method. One of the Cohen methods resulted in a failure rate of 0 % to 10.4 %

Interpretation: In our data material, an absolute limit of 65 % for a pass grade can be defended because the failure rate was generally low. Cohen’s methods could be an alternative in medical schools that have a high failure rate or where there are major variations in the failure rate from one year to the next in the same examination in the course of study.