Decoding the contents of visual short-term memory from human visual and parietal cortex

Abstract

How content is stored in the human brain during visual short-term memory (VSTM) is still an open question. Different theories postulate storage of remembered stimuli in prefrontal, parietal, or visual areas. Aiming at a distinction between these theories, we investigated the content-specificity of BOLD signals from various brain regions during a VSTM task using multivariate pattern classification. To participate in memory maintenance, candidate regions would need to have information about the different contents held in memory. We identified two brain regions where local patterns of fMRI signals represented the remembered content. Apart from the previously established storage in visual areas, we also discovered an area in the posterior parietal cortex where activity patterns allowed us to decode the specific stimuli held in memory. Our results demonstrate that storage in VSTM extends beyond visual areas, but no frontal regions were found. Thus, while frontal and parietal areas typically coactivate during VSTM, maintenance of content in the frontoparietal network might be limited to parietal cortex.

Figure 1.

Delayed similarity detection task using complex artificial stimuli. A, In each trial, subjects were presented with two sample stimuli followed by a retro-cue. This cue indicated which of the two samples was supposed to be remembered. It was surrounded by a mask. After an extended delay, two test stimuli were shown. Subjects were asked to indicate which of the two test stimuli was more similar to the memorized sample. B, All subjects remembered a unique set of four randomly generated samples. C, Stimuli from an example trial. Arrows indicates which test item is more similar to a given sample. For presentation purposes, the correlation between remembered sample and target shown here is higher than in the actual experiment. D, Behavioral performance across subjects. Red squares indicate mean accuracy per subject (N = 17) and red circles show the performance in each of the four runs.

Figure 2.

Brain regions encoding the content of visual working memory. A, B, Localization of working memory content-related information shown in a 3D rendering and in sagittal, coronal, and axial slices. Green areas indicate brain regions that carry significant information (N = 17, p_FWE < 0.05, k = 20) about the stimulus encoded in a given trial. Colored crosses in A indicate the positioning of the slices shown in B. These MNI coordinates are reported in the respective colors in B.

Figure 3.

Distribution of decoding accuracy across pairwise comparisons between memorized samples. The three displays correspond to the three peak voxels reported in the text. Top, Distributions of decoding accuracy across all 102 pairwise comparisons (17 subjects × 6 pairwise comparisons per subject) at the peaks of the respective time series. Bottom, The same data for each subject separately as open circles and the subject-mean as filled squares.

Figure 4.

Time courses of working memory content-related information. Decoding accuracy (in percentages) is plotted against time (seconds) locked to the beginning of the delay phase. Data are shown for the statistical peaks in the three clusters shown in Figure 2A. The delay phase is represented by a blue overlay. Error bars indicate the SEM across subjects. The time series data are shown at the exact time points of data acquisition of the respective slices. The slice acquisition offsets relative to image acquisition onsets was 1.462 s (SEM: 18 ms) in left PPC, 1.58 s (SEM: 16 ms) in right PPC, and 0.620 s (SEM: 22 ms) in right visual cortex.