A Novel Multiple Choice Question Generation Strategy: Alternative Uses for Controlled Vocabulary Thesauri in Biomedical-Sciences Education

Abstract

Multiple choice questions play an important role in training and evaluating biomedical science students. However, the resource intensive nature of question generation limits their open availability, reducing their contribution to evaluation purposes mainly. Although applied-knowledge questions require a complex formulation process, the creation of concrete-knowledge questions (i.e., definitions, associations) could be assisted by the use of informatics methods. We envisioned a novel and simple algorithm that exploits validated knowledge repositories and generates concrete-knowledge questions by leveraging concepts' relationships. In this manuscript we present the development and validation of a prototype which successfully produced meaningful concrete-knowledge questions, opening new applications for existing knowledge repositories, potentially benefiting students of all biomedical sciences disciplines.

Diverging stacked bar charts of survey results. This visualization easily allows to identify the skew between total positive and negative responses, due to the central base. Each question is represented as a row. The total width of the bar shows the percentage of respondents who have non-neutral feelings towards the statement. The depth of color represents the intensity of feeling.

Figure 1:

Definition-type question generation process. Our idea consist on selecting a random node within the category of interest, expose the definition as the question, and retrieve siblings’ names to populate the distractors.

Figure 2:

Desktop view of a question generated from the “Diseases” category, particularly from the “Eye diseases” subsection. The user can see that the topic of the definition corresponds to “Ocular motility disorders”, and is presented with four possible alternatives for the given definition.

Figure 3:

Mobile view of feedback received when a question is answered wrong.

Figure 4:

Composition of MeSH. The central pie chart represents the relative contribution of each category to the tree. The orbital graphs represent each category composition (distinct number of subcategories, and distinct levels of depth). For example, “Geographicals” has only one subcategory, with 8 levels of depth, while “Organisms” has 5 subcategories, varying from 4 levels down to 11 levels of depth). The size of a node represents the relative number of concepts in that level.