A comparison of clinical-scenario (case cluster) versus stand-alone multiple choice questions in a problem-based learning environment in undergraduate medicine

Abstract

Objectives: To compare stand-alone multiple choice questions (MCQs) and integrated clinical-scenario (case cluster) multiple choice questions (CS-MCQs) in a problem-based learning (PBL) environment.

Methods: A retrospective descriptive analysis of MCQ examinations was conducted in a course that integrates the subspecialties of anatomical pathology, chemical pathology, hematology, immunology, microbiology and pharmacology. The MCQ items were analyzed for their reliability (Kuder-Richardson-20, KR-20), level of difficulty (Pi), discrimination index (Di), item distractors and student performances. The statistical analysis of the results was extracted from the integrity online item-analysis programme. The results of the standard stand-alone and CS multiple choice questions were compared.

Results: KR-20 for the CS-MCQs and stand-alone MCQs was consistently high. KR-20 and Pi were higher for the CS-MCQs. There was no significant difference between the CS-MCQs and stand-alone MCQs in Pi and Di. A range of difficulty levels was found based on Bloom's taxonomy. The mean scores for the class were higher for the CS-MCQ examination. The compilation of the CS-MCQ examination was more challenging.

Conclusions: CS-MCQs compare favorably to stand-alone MCQs and provide opportunities for the integration of sub-specialties and assessment in keeping with PBL. They assess students' cognitive skills and are reliable and practical. Different levels of item difficulty promote multi-logical and critical thinking. Students' scores were higher for the CS-MCQ examination, which may suggest better understanding of the material and/or better question clarity. The scenarios have to flow logically. Increasing the number of scenarios ensures the examination of more course content.

Keywords: APS-I, Applied Para-clinical Sciences-I; APS-II, Applied Para-clinical Sciences-II; APS-III, Applied Para-clinical Sciences-III; CA, continuous assessment; CPBR, corrected point-biserial ratio; CS, clinical scenario; CS-MCQ, clinical scenario multiple choice question; Clinical scenario; Difficulty; Discrimination; EMQ, extended matching questions; Integration; KR-20, KuderRichardson-20; MCQ, multiple choice questions; MEQ, modified essay questions; PBL; PBL, problem based learning; PDQ, progressive disclosure questions; SAQ, short answer questions.

Figure 1

APS-III: Item Discrimination/Difficulty: Year 2011–2012 (Case clusters-item1-67, stand-alone: item 68–75).

Figure 2

APS-III: Item discrimination/difficulty: Year 2012–2013 (Case clusters-Item 1–63, stand alone: Item 64–75).

Figure 3

APS-III: Item discrimination/difficulty Year 2013–2014: (Case clusters-item 1–58, stand alone: item 59–75).