Summative assessments are more powerful drivers of student learning than resource intensive teaching formats

Abstract

Background: Electrocardiogram (ECG) interpretation is a core clinical skill that needs to be acquired during undergraduate medical education. Intensive teaching is generally assumed to produce more favorable learning outcomes, but recent research suggests that examinations are more powerful drivers of student learning than instructional format. This study assessed the differential contribution of teaching format and examination consequences to learning outcome regarding ECG interpretation skills in undergraduate medical students.

Methods: A total of 534 fourth-year medical students participated in a six-group (two sets of three), partially randomized trial. Students received three levels of teaching intensity: self-directed learning (two groups), lectures (two groups) or small-group peer teaching facilitated by more advanced students (two groups). One of the two groups on each level of teaching intensity was assessed in a formative, the other in a summative written ECG examination, which provided a maximum of 1% credit points of the total curriculum. The formative examination provided individual feedback without credit points. Main outcome was the correct identification of ≥3 out of 5 diagnoses in original ECG tracings. Secondary outcome measures were time spent on independent study and use of additional study material.

Results: Compared with formative assessments, summative assessments increased the odds of correctly identifying at least three out of five ECG diagnoses (OR 5.14; 95% CI 3.26 to 8.09), of spending at least 2 h/week extra on ECG self-study (OR 4.02; 95% CI 2.65 to 6.12) and of using additional learning material (OR 2.86; 95% CI 1.92 to 4.24). Lectures and peer teaching were associated with increased learning effort only, but did not augment examination performance.

Conclusions: Medical educators need to be aware of the paramount role of summative assessments in promoting student learning. Consequently, examinations within medical schools need to be closely matched to the desired learning outcomes. Shifting resources from implementing innovative and costly teaching formats to designing more high-quality summative examinations warrants further investigation.

Figure 1

Schematic diagram of the study design. The six study groups differed with regards to assessment consequences (summative/formative) and teaching format (self-directed learning/lectures/small-group peer teaching).

Figure 2

Descriptive analysis of primary and secondary outcomes. The figure shows percentages of students correctly identifying at least three out of five diagnoses in the electrocardiogram (ECG) exit examination (black columns), self-reporting to have spent more than 2 h/week on independent ECG self-study (dark gray columns) and of having used additional ECG learning material during the module (light gray columns) by study group. Error bars indicate 95% confidence intervals of prevalence estimates.