Listening for recollection: a multi-voxel pattern analysis of recognition memory retrieval strategies

Abstract

Recent studies of recognition memory indicate that subjects can strategically vary how much they rely on recollection of specific details vs. feelings of familiarity when making recognition judgments. One possible explanation of these results is that subjects can establish an internally directed attentional state ("listening for recollection") that enhances retrieval of studied details; fluctuations in this attentional state over time should be associated with fluctuations in subjects' recognition behavior. In this study, we used multi-voxel pattern analysis of fMRI data to identify brain regions that are involved in listening for recollection. We looked for brain regions that met the following criteria: (1) Distinct neural patterns should be present when subjects are instructed to rely on recollection vs. familiarity, and (2) fluctuations in these neural patterns should be related to recognition behavior in the manner predicted by dual-process theories of recognition: Specifically, the presence of the recollection pattern during the pre-stimulus interval (indicating that subjects are "listening for recollection" at that moment) should be associated with a selective decrease in false alarms to related lures. We found that pre-stimulus activity in the right supramarginal gyrus met all of these criteria, suggesting that this region proactively establishes an internally directed attentional state that fosters recollection. We also found other regions (e.g., left middle temporal gyrus) where the pattern of neural activity was related to subjects' responding to related lures after stimulus onset (but not before), suggesting that these regions implement processes that are engaged in a reactive fashion to boost recollection.

Keywords: episodic memory; fMRI; long-term memory; pattern classification.

Figure 1

Schematic overview of the experiment. Different experimental tasks were performed in Phase 1 and Phase 2 of the experiment. Scanning was performed in both phases. The classifier was trained on Phase 1 data to distinguish between brain activity from recollection blocks vs. familiarity blocks. Next, the trained classifier was applied to Phase 2 data, in order to estimate the subject's use of recollection vs. familiarity at each time point during the plurals recognition task. Classifier outputs were then related to the subject's behavior in the plurals task.

Figure 2

The block and event sequence for one Phase 1 study-test run. Each run began with 10 s of fixation followed by a 66-s study phase. The study phase was followed by an alternating sequence of 22-s fixation periods and 32-s recognition test blocks performed under either familiarity or recollection instructions. In each run, one of the two block-types (recollection or familiarity test instructions) appeared as the first and fourth block, and the other type appeared as the second and third. The arrangements of recollection and familiarity blocks varied between these two sets of positions across runs. Within each block, the first 6 s consisted of a cue indicating the condition, followed by a 2-s fixation before the 12 consecutive test trials with onsets every 2 s.

Figure 3

Event sequence during two sample trials of a Phase 2 test run (the timeline for the entire test run is shown at the bottom of the figure). Test trials were presented for 2 s, followed by a response feedback and fixation period of jittered duration, varying between 2 and 10 s. A running point total was visible at all times during the test period. After subjects entered their response, the current award or penalty was shown (along with a fixation cross) and the total was updated.

Figure 4

Sphere clusters passing our statistical tests for the MVPA analysis at family-wise error rate <0.05. Values plotted at each voxel are the average number of significant spheres in which the voxel was included, scaled from red to yellow, with yellow regions indicating that a voxel appeared in an average of 10 or more significant spheres. Sphere results were computed separately for four time windows relative to the test stimulus onset.

Figure 5

Sphere clusters passing our statistical tests for the AVG analysis at family-wise error rate <0.05. Values plotted at each voxel are the average number of significant spheres in which the voxel was included, scaled from red to yellow, with yellow regions indicating that a voxel appeared in an average of 10 or more significant spheres. Sphere results were computed separately for four time windows relative to the test stimulus onset.

Figure 6

Classifier output as a function of recognition behavior in the right supramarginal gyrus/BA 40 for four time windows around the trial onset. The values shown here were computed by averaging classifier output from all of the spheres belonging to the cluster. The left panel shows MVPA results and the right panel shows AVG results. Each panel shows D_RL (the classifier output difference for related-lure correct rejections vs. false alarms) in red and D_STUDIED (the classifier output difference for studied-item misses vs. hits) in blue. For both measures, positive values indicate greater use of recollection (according to the classifier) for “no” responses (correct rejections and misses) than for “yes” responses (false alarms and hits). Asterisks indicate time windows where the relationship between classifier output and behavior was significant (according to the non-parametric statistical tests described in the text). Error bars show the standard error of the mean across subjects.

Figure 7

Classifier output as a function of recognition behavior in the left cuneus/precuneus region for four time windows around the trial onset, for the MVPA and AVG analyses. See the caption of Figure 6 for explanation of the plots.

Figure 8

Classifier output as a function of recognition behavior in the left posterior parahippocampal region for four time windows around the trial onset, for the MVPA and AVG analyses, See the caption of Figure 6 for explanation of the plots.

Figure 9

Classifier output as a function of recognition behavior in the left lingual gyrus for four time windows around the trial onset, for the MVPA and AVG analyses. See the caption of Figure 6 for explanation of the plots.

Figure 10

Classifier output as a function of recognition behavior in the left middle temporal gyrus for four time windows around the trial onset, for the MVPA and AVG analyses. See the caption of Figure 6 for explanation of the plots.

Figure 11

Classifier output as a function of recognition behavior in the left middle frontal gyrus for four time windows around the trial onset, for the MVPA and AVG analyses. See the caption of Figure 6 for explanation of the plots.