Decoding the content of visual short-term memory under distraction in occipital and parietal areas

Abstract

Recent studies have provided conflicting accounts regarding where in the human brain visual short-term memory (VSTM) content is stored, with strong univariate fMRI responses being reported in superior intraparietal sulcus (IPS), but robust multivariate decoding being reported in occipital cortex. Given the continuous influx of information in everyday vision, VSTM storage under distraction is often required. We found that neither distractor presence nor predictability during the memory delay affected behavioral performance. Similarly, superior IPS exhibited consistent decoding of VSTM content across all distractor manipulations and had multivariate responses that closely tracked behavioral VSTM performance. However, occipital decoding of VSTM content was substantially modulated by distractor presence and predictability. Furthermore, we found no effect of target-distractor similarity on VSTM behavioral performance, further challenging the role of sensory regions in VSTM storage. Overall, consistent with previous univariate findings, our results indicate that superior IPS, but not occipital cortex, has a central role in VSTM storage.

Figure 1

Main experimental task from Experiments 1 and 3. Participants were shown two orientated gratings, and then cued as to which to remember. The cue presented here is enlarged for clarity. After a long delay, a third grating appeared and they were asked to judge whether this grating was jittered clockwise or counterclockwise to the remembered grating. During the delay, participants either saw a blank screen with a fixation dot (trials without distractors) or a sequential presentation of task irrelevant faces or gazebos (trials with distractors). In Experiment 1, trials without distractors were presented in the first half of the experiment while those with distractors were presented in the second half, making distractor presence/absence predictable. In Experiment 3, the two types of trials were randomly intermixed within a run, making distractor presence/absence unpredictable.

Figure 2

ROIs and the localizer tasks. A moving, flashing, colored checkerboard wedge (a) and an object-based VSTM task (b) were used to define occipital and parietal topographic regions (c) and superior IPS (d), respectively. In the VSTM task, participants were shown a sequential presentation of either 1, 2, 3, 4, or 6 real world objects at fixation, and, after a brief delay, reported whether the test object shown at fixation was a match or non-match to one of the remembered objects. Superior IPS was defined as a region that tracked the behavioral VSTM capacity measures in this task. IPL and SPL (e) were anatomically defined. Each ROI was further refined to select voxels that respond to the task stimuli. All ROIs are shown here on the inflated left hemisphere of an example participant.

Figure 3

MVPA decoding accuracy for the average VSTM delay period activity in V1–V4 (a) and superior IPS (b) in Experiment 1 (with predictable distractors) and Experiment 3 (with unpredictable distractors). The same ten participants took part in both experiments. Although the presence and predictability of distractors did not impact behavioral performance, when the presence of distractors was predictable in Experiment 1, V1–V4 showed successful VSTM decoding when distractors were absent but a significant drop to chance-level decoding when distractors were present. However, when the presence of distractors was unpredictable in Experiment 3, V1–V4 showed weaker but significant and comparable VSTM decoding for both distractor present and absent conditions. Unlike V1–V4, superior IPS mirrored behavioral performance and showed consistent and significant VSTM decoding irrespective of distractor presence and predictability. Error bars indicate s.e.m. * p < 0.05; ** p < 0.01; *** p < 0.001; ns, non-significant; No dist, trials without distractors; Dist, trials with distractors.

Figure 4

Stimuli and task for Experiment 2. Participants were continuously shown a low contrast oriented grating that was either presented alone (trials without distractors) or overlaid with stronger distractor stimuli that flickered on and off following the distractor presentation timing during the delay period in Experiment 1 (trials with distractors). Participants performed a 1-back letter repetition detection task at fixation. Both the fixation dot and the letter have been enlarged in this figure for clarity.

Figure 5

MVPA decoding results for Experiment 2. Eight of the ten participants from Experiments 1 and 3 took part in this experiment. Decoding accuracy for the presented grating was significantly above chance in both trials with and without distractors, with no differences seen between trial types, suggesting that MVPA can decode two simultaneously represented stimuli in occipital cortex. Error bars indicate s.e.m. ns non-significant; * p < 0.05; ** p < 0.01; No dist, trials without distractors; Dist, trials with distractors.

Figure 6

Correlation of neural and behavioral VSTM representations from Experiment 4. Six participants from Experiment 1 took part in this experiment. Both V1–V4 (a) and superior IPS (b) show strong negative correlations between behavioral (RT) and neural (decoding accuracy) measures of VSTM representation similarity across the six orientations tested, showing that the more similar a pair of orientation representations are in these brain regions during the VSTM delay period, the harder it is to discriminate them behaviorally in a change-detection task. In V1–V4, two pairs of orientation representations (40º to 160º and 130º to 160º) had identical RTs and decoding accuracies, and so both points occupy the same place in the graph. These results establish a significant link between VSTM representations in both brain regions and behavioral VSTM performance when distractors were absent during the delay period.

Figure 7

Accuracy results for Experiment 5. Six participants from Experiment 1 took part in this experiment. In this behavioral experiment, the presence and absence of distractors during the VSTM delay period as well as the similarity between the target and distractors were varied. There was no difference in accuracy, as measured by percent correct for any distractor condition, nor did any distractor condition differ from the no distractor condition, showing that neither distractor presence/absence nor target-distractor similarity affected performance. Error bars indicate s.e.m. No dist, trials without distractors, Faces, trials with face distractors during the delay, Gazebos, trials with gazebo distractors, Gratings, trials with oriented grating distractors.