Re-examining the testing effect as a learning strategy: the advantage of retrieval practice over concept mapping as a methodological artifact

Abstract

Several previous studies appear to have demonstrated that studying with retrieval practice produces more learning than studying with concept mapping, a finding based on which an extended use of retrieval practice in educational practice was recommended. However, a closer examination of the methods used in these previous studies reveals a crucial confounding variable: Whereas participants in the concept mapping conditions performed a concept mapping task without any subsequent memorizing of the learning material, participants in the retrieval practice conditions performed not only retrieval practice but also an additional memorization task, which doubled the total memorization time. The present preregistered study examined whether the advantage observed in the retrieval practice condition over the concept mapping condition in previous studies was actually driven by additional memorization rather than by retrieval practice. While we replicated the previous finding that retrieval practice in combination with additional memorizing produces more learning than concept mapping without additional memorizing, this advantage of retrieval practice over concept mapping vanished when participants in the concept mapping condition, too, memorized the learning material after having created a concept map. These findings demonstrate that the assumed advantage of retrieval practice over concept mapping in fact represents a methodological artifact. Besides serving as a reminder of the importance of a solid methodology, the present study also illustrates the importance of using of an adequate terminology. Depicting a learning strategy condition as "retrieval practice" when the condition actually encompasses not only retrieval practice but also additional memorizing obfuscates the possibility that observed memory advantages may not be fueled by retrieval practice, i.e., the learning strategy as such. We conclude by giving an outlook on the ramifications of our findings for cognitive and educational psychology.

Keywords: cognitive psychology; concept mapping; educational psychology; learning strategies; retrieval practice; testing effect.

FIGURE 1

Illustration of the confounding variable “Memorization” in Karpicke and Blunt’s (2011) study. The terms used to describe the different learning strategy conditions, namely, “retrieval practice” and “concept mapping,” give the impression that only retrieval practice or concept mapping, respectively, were performed in each condition. However, in the so-called “retrieval practice” condition, participants not only performed a retrieval task but also an additional memorization task. By contrast, in the concept mapping condition, participants only performed a concept mapping task without any additional memorization of the learning material. The additional memorization task doubled the time participants spent memorizing the learning material for the later test in the retrieval practice condition. Note that the text the participants were to learn was available during the creation of the concept map but not during retrieval practice.

FIGURE 2

Illustration of the four learning strategy conditions. The “Retrieval Practice with Additional Memorization and with Additional Instruction ‘Recall as Much as Possible’ Condition” (RP + AM + AI) and the “Concept Mapping without Additional Memorization and without Additional Instruction ‘Incorporate as Much as Possible’ Condition” (CM – AM – AI) are exact replications of the conditions examined by Karpicke and Blunt (2011), i.e., RP + AM + AI = Karpicke and Blunt’s “retrieval practice condition”; CM – AM – AI = Karpicke and Blunt’s “concept mapping condition.” In the “Concept Mapping without Additional Memorization and with Additional Instruction ‘Incorporate as Much as Possible’ Condition” (CM – AM + AI), to control for the confounder of different instructions, participants were prompted during the creation of the concept map as well that the concept map should contain as many details of the text as possible. In the “Concept Mapping with Additional Memorization and with Additional Instruction ‘Incorporate as Much as Possible’ Condition” (CM + AM + AI), to additionally control for the confounder of additional memorization, participants were asked to memorize the material after the creation of the concept map as well. Note that the text that the participants were to learn was available during the creation of the concept map but not during retrieval practice.

FIGURE 3

Memory Performance. The proportion of correct answers for verbatim questions (A) and inference questions (B) is shown as a function of the four learning strategy conditions (Retrieval Practice with Additional Memorization and with Additional Instruction “Recall as Much as Possible,” RP + AM + AI; Concept Mapping without Additional Memorization and without Additional Instruction “Incorporate as Much as Possible,” CM – AM – AI; Concept Mapping without Additional Memorization and with Additional Instruction “Incorporate as Much as Possible,” CM – AM + AI; Concept Mapping with Additional Memorization and with Additional Instruction “Incorporate as Much as Possible,” CM + AM + AI). The violin plots show the probability density across participants; data points are plotted as dots. Center horizontal line markers show the medians. Box limits indicate the 25th and 75th percentiles. Whiskers extend 1.5 times the interquartile range from the 25th and 75th percentiles.

FIGURE 4

Illustration of the divergent rationalities underlying real-life learning and experimental research. Although the domains overlap, the focus of the questions asked is different: determining the optimal combination of cognitive processes (real-life learning) vs. determining the specific effect of isolated cognitive processes (experimental research). As shown on the right side, this problem may be obfuscated by the use of imprecise terminology. If the term “retrieval practice” is used to delineate a learning strategy which is actually a combination of retrieval practice and restudying, this may lead to results that may seem surprising and informative (e.g., “retrieval practice is better than restudying”) at first glance, although they are actually rather trivial (e.g., “retrieval practice plus restudying is better than restudying alone”). Consequently, potential implications for education drawn on the basis of experimental laboratory studies should be considered with caution as overemphasizing one factor or an oversimplified transfer to real-life learning may lead to already existing knowledge on learning being neglected.