The influence of question design on the response to self-assessment in www.elearnSCI.org: a submodule pilot study

Abstract

Study design: This is an interventional training session.

Objective: The objective of this study was to investigate the difference in response to self-assessment questions in the original and an adjusted version for a submodule of www.elearnSCI.org for student nurses.

Setting: The study was conducted in a teaching hospital affiliated to Peking University, China.

Methods: In all, 28 student nurses divided into two groups (groups A and B; 14 in each) received a print-out of a Chinese translation of the slides from the 'Maintaining skin integrity following spinal cord injury' submodule in www.elearnSCI.org for self-study. Both groups were then tested using the 10 self-assessment multiple-choice questions (MCQs) related to the same submodule. Group A used the original questions, whereas group B received an adjusted questionnaire.

Results: The responses to four conventional single-answer MCQs were nearly all correct in both groups. However, in three questions, group A, with the option 'All of the above', had a higher number of correct answers than group B, with multiple-answer MCQs. In addition, in another three questions, group A, using the original multiple-answer MCQs, had fewer correct answers than group B, where it was only necessary to tick a single incorrect answer.

Conclusion: Variations in design influence the response to questions. The use of conventional single-answer MCQs should be reconsidered, as they only examine the recall of isolated knowledge facts. The 'All of the above' option should be avoided because it would increase the number of correct answers arrived at by guessing. When using multiple-answer MCQs, it is recommended that the questions asked should be in accordance with the content within the www.elearnSCI.org.