Using cognitive load theory to evaluate and improve preparatory materials and study time for the flipped classroom

Abstract

Background: Preclinical medical education is content-dense and time-constrained. Flipped classroom approaches promote durable learning, but challenges with unsatisfactory student preparation and high workload remain. Cognitive load theory defines instructional design as "efficient" if learners can master the presented concepts without cognitive overload. We created a PReparatory Evaluation Process (PREP) to systematically assess and measure improvement in the cognitive-load efficiency of preparatory materials and impact on study time (time-efficiency).

Methods: We conducted this study in a flipped, multidisciplinary course for ~ 170 first year students at Harvard Medical School using a naturalistic post-test design. For each flipped session (n = 97), we assessed cognitive load and preparatory study time by administering a 3-item PREP survey embedded within a short subject-matter quiz students completed before class. Over three years (2017-2019), we evaluated cognitive load- and time- based efficiency to guide iterative revisions of the materials by content experts. The ability of PREP to detect changes to the instructional design (sensitivity) was validated through a manual audit of the materials.

Results: The average survey response rate was ≥ 94%. Content expertise was not required to interpret PREP data. Initially students did not necessarily allocate the most study time to the most difficult content. Over time, the iterative changes in instructional design increased the cognitive load- and time-based efficiency of preparatory materials with large effect sizes (p < .01). Furthermore, this increased the overall alignment of cognitive load with study time: students allocated more time to difficult content away from more familiar, less difficult content without increasing workload overall.

Conclusions: Cognitive load and time constraints are important parameters to consider when designing curricula. The PREP process is learner-centered, grounded in educational theory, and works independently of content knowledge. It can provide rich and actionable insights into instructional design of flipped classes not captured by traditional satisfaction-based evaluations.

Keywords: Cognitive load theory; Educational quality improvement; Efficiency; Flipped classroom; Instructional design.

Fig. 1

A) Efficiency graphs (EG). To produce EGs, the data were standardized by student (z-scores), aggregated by session, and mean session values were plotted. Sessions above the y = x line were considered more efficient, sessions below the line less efficient. The position on the graph with respect to the line can also be expressed as efficiency metric E = (y – x)/√2 [23] B) Comparing cognitive load- and time-based efficiency in year 1. Each dot represents one session. Sessions were ordered alphabetically and then numbered from 1–97. To better visualize the position of each session with respect to the line, we colored each dot with the value of the efficiency metric E, for time or cognitive load respectively. In year 1, 25 out of 97 sessions were very efficient (E ≥ 0.5) in either time (n = 7), cognitive load (n = 7), or both (n = 11). Similarly, 27 out 97 sessions were quite inefficient (defined as E ≤ − 0.5) in either time (n = 9), or cognitive load (n = 11), or both (n = 7) C) Alignment of prep time with cognitive load over the years. Cognitive load based EGs for year 1 and year 3 were plotted. Each dot represents one session color-coded by prep time in hours. Graphs show a change from year 1 to year 3 in better alignment of prep time with most difficult materials D) EGs with Cluster overlay. Cognitive load- and time-based efficiency graphs from panel B were overlayed with the cluster denomination. E) and F) Examples of iterative changes to individual sessions from year 1 to 3 in two different disciplines. The trail line illustrates the change in position on the graph over the years. The line starts with year 1. The end position in year 3 is indicated by the circular marker.

Fig. 2

A) Familiarity. Familiarity ratings plotted versus difficulty and overlaid with the cluster denomination. The more familiar cluster 2 sessions stood out as a group with moderate difficulty. Cluster 1 and 3 sessions both covered content unfamiliar to the students from prior courses but greatly differed in perceived difficulty of the content B) Course design. Sessions were plotted by cluster in the order of occurrence over the time of the course. (Please note that the numbers do NOT correspond to the labels in Fig. 1). Cluster 1 represents content that is unfamiliar and least difficult. Cluster 2 content is most familiar, and moderately difficult. Cluster 3 content is most difficult and least familiar. Preparation times differ across clusters and are discussed in more depth in the text. The course progresses from more familiar to less familiar content over time. Over the years the number of cluster 1 sessions slightly increased (not statistically significant) and cluster 3 sessions were intentionally distributed more evenly across weeks to balance the weekly workload. (A week comprises 8–11 sessions). C) Impact. Cognitive load efficiency graphs overlaid with prep time as contour plot highlight how students increasingly invest their time in the most difficult concepts over the years.