Collusion detection in multiple choice examinations

Abstract

Objective: To develop and test a novel method for collusion detection in multiple choice examinations.

Subjects and methods: Answers from two negatively marked medical prize examinations for two different years were analysed. Both examinations were administered electronically. One examination was formally invigilated, while the second was not; instead, candidates were able to sit the test at any time and at any computer with an internet connection. We examine pairs of students and compare correlations between their answers. Our approach allows us to correct for the difficulty of individual questions and for the estimated ability and the degree of risk aversion of the students. We compare the results of this statistical analysis with other information on the timing of the answers and the physical location of the computer, both of which are available to the web-server.

Results: Significant correlation between several candidates who either admitted having cheated or could be linked to other corroborating evidence of collusion was found.

Conclusion: It is possible to detect collusion in multiple choice examinations in a statistical way by examining the patterns of answers between pairs of candidates. In examinations that are delivered on-line, information is often available on the location of the candidates and the timings of their answers, and can be used as additional corroborative evidence.