Neurofeedback helps to reveal a relationship between context reinstatement and memory retrieval

Abstract

Theories of mental context and memory posit that successful mental context reinstatement enables better retrieval of memories from the same context, at the expense of memories from other contexts. To test this hypothesis, we had participants study lists of words, interleaved with task-irrelevant images from one category (e.g., scenes). Following encoding, participants were cued to mentally reinstate the context associated with a particular list, by thinking about the images that had appeared between the words. We measured context reinstatement by applying multivariate pattern classifiers to fMRI, and related this to performance on a free recall test that followed immediately afterwards. To increase sensitivity, we used a closed-loop neurofeedback procedure, whereby higher classifier evidence for the cued category elicited increased visibility of the images from the studied context onscreen. Our goal was to create a positive feedback loop that amplified small fluctuations in mental context reinstatement, making it easier to experimentally detect a relationship between context reinstatement and recall. As predicted, we found that greater amounts of classifier evidence were associated with better recall of words from the reinstated context, and worse recall of words from a different context. In a second experiment, we assessed the role of neurofeedback in identifying this brain-behavior relationship by presenting context images again and manipulating whether their visibility depended on classifier evidence. When neurofeedback was removed, the relationship between classifier evidence and memory retrieval disappeared. Together, these findings demonstrate a clear effect of context reinstatement on memory recall and suggest that neurofeedback can be a useful tool for characterizing brain-behavior relationships.

Keywords: Free recall; Long-term memory; Real-time fMRI.

Figure 1

Study procedure. An example run of the task in Experiment 1. Each run was composed of encoding, context reinstatement, and recall phases. During the encoding phase, participants studied two lists of sequentially presented words, Lists A & B. Each of the lists was embedded in a different context by interleaving the words with images of a single category (scenes or faces). During the context reinstatement phase, participants were provided with a list name (e.g., List B) as a cue for which context (either scenes or faces) to reinstate. Participants were presented with composite face/scene images, initialized at 50% face and 50% scene. This mixture proportion was adjusted during the context reinstatement period to reflect the real-time decoding evidence for the cued context. The top row shows representative composite images. The middle row shows the corresponding proportion of the cued category of the composite image. The bottom row shows the real-time classifier evidence for the cued minus the uncued category for each TR during the context reinstatement period. Greater evidence for the cued context resulted in more of that category in the composite image (and less evidence resulted in less of that category). During the recall phase, participants were presented with a list name as a memory probe. Then, they were instructed to freely recall as many words as possible from the probed list. In validly cued runs (6 of 8 runs, 75%), the memory probe was the same as the context cue. In invalidly cued runs (2 of 8 runs, 25%), the memory probe was different from the context cue

Figure 2.

Timecourse of real-time multivariate classifier decoding of context. The average classifier evidence for each participant across all feedback runs is plotted in thin gray lines. The average timecourse across participants is plotted in black, with the gray ribbon indicating the standard error of the mean. The y-axis shows the classifier evidence for the cued category minus the classifier evidence of the uncued category. The x-axis shows the number of TRs (1.5s) during the context reinstatement phase.

Figure 3.

Effects of context reinstatement cue validity on memory in Experiment 1. (a) Memory recall performance. For valid runs, the cue at the start of the context reinstatement period matched the memory probe at the start of the free recall period. For invalid runs, the cue did not match the memory probe. Each gray dot indicates the average number of words recalled per participant (n=24). The height of the bar indicates the population average, and the error bars indicate the standard error of the mean. Memory performance was enhanced following valid cues (* p<0.05). (b) To quantify the relationship between classifier evidence and recall, we computed (across runs, within each participant) the linear fit between classifier evidence and recall, separately for valid runs and invalid runs. Statistics were computed using the slopes of the linear fits per condition. Each dot corresponds to the slope of the linear fit in a single condition (valid or invalid) for each participant. The height of the bar indicates the population average, and the error bars indicate the standard error of the mean. The slope relating classifier evidence to behavior differed between valid and invalid runs (*** p<0.001). (c) For validly cued runs, the amount of context reinstatement positively related to the number of recalls (p<0.01). That is, there was a reliably positive relationship between the evidence for the cued context minus uncued context (x-axis) and the total number of recalls (y-axis) for each run. The linear fit across runs within a single participant is depicted as a gray line. The mean linear fit is depicted in teal. (d) In invalidly cued runs, the amount of context reinstatement negatively related to the number of recalls (p<0.01). The linear fit across runs within a single participant is depicted as a gray line. The mean linear fit is depicted in orange.

Figure 4

Simulating feedback. (a) A schematic of the hypothesized context reinstatement process with the link mediated by real-time neurofeedback in orange. Classifier evidence is jointly determined by internal mental contextual reinstatement and external perceptual evidence for context. Neurofeedback “closes the loop” by allowing classifier evidence to influence perceptual evidence (b) Results of computational simulations of the correlation between classifier evidence and memory recall behavior. Simulations were completed for various manipulations of the feedback as well as perceptual evidence: real-time neurofeedback (in which the perceptual evidence reflects the classifier evidence for the cued category), maximal perceptual input (100% cued category, 0% uncued category), balanced perceptual input (50% cued category, 50% uncued category), no perceptual input (0% cued, 0% uncued), inverted real-time neurofeedback (in which the perceptual evidence reflects the classifier evidence for the uncued category), and yoked-control feedback (in which the perceptual evidence reflects the classifier evidence from another run. Each violin plot represents the correlations computed across 10,000 simulations. The mean correlation is depicted in the horizontal black line, and 95% CIs in the vertical black line.

Figure 5

Feedback mediates the link between context reinstatement and memory in Experiment 2. (a) Memory recall performance. In feedback runs, validly-cued feedback was provided during the context reinstatement period. In non-feedback control feedback runs, there was no real-time feedback during the context reinstatement period Each dot corresponds to the average number of recalls for a participant (n=24). The height of the bar indicates the population average. The error bars indicate standard error of the mean. Memory performance did not reliably differ between these feedback and non-feedback conditions (p>0.1). (b) To quantify the relationship between classifier evidence and recall, we computed (across runs, within each participant) the linear fit between classifier evidence and recall, separately for feedback runs and non-feedback runs. Statistics were computed using the slopes of the linear fits per condition. Each dot corresponds to the slope of the linear fit in a single condition (feedback or no feedback) for each participant. The relationship between context reinstatement and memory performance was reliably greater in the feedback condition than in the non-feedback condition (** p<0.01). (c) Evidence for the cued context in the feedback condition was positively related to the number of recalls, replicating the effect in the valid feedback condition of Experiment 1 (p<0.05). The linear fit for each participant is depicted as a gray line. The teal line is the mean fit across the population. (d) Evidence for the cued context in the non-feedback control condition was not positively related to memory recall performance (p>0.1). The black line is the mean fit across the population.