Visual Working Memory Is Independent of the Cortical Spacing Between Memoranda

Abstract

The sensory recruitment hypothesis states that visual short-term memory is maintained in the same visual cortical areas that initially encode a stimulus' features. Although it is well established that the distance between features in visual cortex determines their visibility, a limitation known as crowding, it is unknown whether short-term memory is similarly constrained by the cortical spacing of memory items. Here, we investigated whether the cortical spacing between sequentially presented memoranda affects the fidelity of memory in humans (of both sexes). In a first experiment, we varied cortical spacing by taking advantage of the log-scaling of visual cortex with eccentricity, presenting memoranda in peripheral vision sequentially along either the radial or tangential visual axis with respect to the fovea. In a second experiment, we presented memoranda sequentially either within or beyond the critical spacing of visual crowding, a distance within which visual features cannot be perceptually distinguished due to their nearby cortical representations. In both experiments and across multiple measures, we found strong evidence that the ability to maintain visual features in memory is unaffected by cortical spacing. These results indicate that the neural architecture underpinning working memory has properties inconsistent with the known behavior of sensory neurons in visual cortex. Instead, the dissociation between perceptual and memory representations supports a role of higher cortical areas such as posterior parietal or prefrontal regions or may involve an as yet unspecified mechanism in visual cortex in which stimulus features are bound to their temporal order.SIGNIFICANCE STATEMENT Although much is known about the resolution with which we can remember visual objects, the cortical representation of items held in short-term memory remains contentious. A popular hypothesis suggests that memory of visual features is maintained via the recruitment of the same neural architecture in sensory cortex that encodes stimuli. We investigated this claim by manipulating the spacing in visual cortex between sequentially presented memoranda such that some items shared cortical representations more than others while preventing perceptual interference between stimuli. We found clear evidence that short-term memory is independent of the intracortical spacing of memoranda, revealing a dissociation between perceptual and memory representations. Our data indicate that working memory relies on different neural mechanisms from sensory perception.

Keywords: cued recall; short-term memory; visual crowding; visual working memory.

Figure 1.

Experiment 1 design. A, Differences in cortical spacing in peripheral vision. Top row, Screen coordinates of stimuli in peripheral vision with respect to the point of fixation (black spot). Bottom row, Inter-item spacing after cortical transformation. Such a cortical representation of space occurs in V1, which is hypothesized to maintain memory representations. Cortically transformed coordinates are normalized to the central target position. Green spots and purple diamonds represent radial and tangential spatial arrangements of stimuli, respectively. Note that, although stimuli are equally spaced in screen coordinates across conditions, radially arranged stimuli have less intracortical spacing than tangentially arranged stimuli. B, Stimulus design. Memoranda were randomly oriented colored bars presented sequentially along either the radial or tangential axis. Note that the center stimulus in each condition occupies the same screen (and therefore cortical) location. C, Example trial sequence. Observers fixated a white spot while memoranda were presented in sequence. Following a delay after the presentation of the third item, a probe was shown matching the color and location of one item chosen at random, cueing observers to move the mouse to report the remembered orientation of that item. A response bar appeared within the circle after the first mouse movement was detected, allowing observers to make their response using a method of adjustment.

Figure 2.

Results of Experiment 1. A, Report variability for each condition. Filled circles show the mean circular SD of reports for radial (green) and tangential (purple) configurations. Colored lines show individual participants' data. Error bars indicate ±1 SE. B, Differences in report variability across conditions. The black datum shows the mean difference and the colored data show individual difference scores, with colors corresponding to lines in A. C, Error distributions and model fit. Frequency of errors for the radial and tangential conditions are expressed as probability densities, with colors as in A. Data are shown for 16 equally spaced bins in the range [−π, π]. The solid blue line shows predictions of the best-fitting model, in which we assume memory is independent of the configuration of stimuli (shaded area indicates ±1 SE).

Figure 3.

Design and results of the crowding task. A, Example trial sequence. After fixating a white spot, three stimuli were presented in the upper visual field. An observer's task was to identify the orientation of the center stimulus and report its orientation by clicking on the matching stimulus in gray in the subsequent display. The distance between target and distractors on each trial was controlled via an adaptive procedure. B, Example results and psychometric functions. Differently colored data show results for two differently performing observers. Solid lines show Weibull functions fit to each dataset. Dashed black lines and colored X symbols show the midpoint of the function and corresponding critical spacing estimates, respectively, for each participant. C, Estimated critical spacing for 19 observers. The median critical spacing is shown as the black datum and individual participants' values are shown in various colors. Estimates corresponding to the psychometric functions in B are shown as X symbols. Data have been jittered randomly on the x-axis to minimize overlap. Error bars indicate ±1 SE.

Figure 4.

Experiment 2 design and results. A, Example trials of the crowded and uncrowded conditions. Observers fixated a white spot and viewed a sequence of randomly orientated memoranda that appeared within (crowded condition) or beyond (uncrowded condition) the critical spacing of their upper visual field, as indicated by the white dotted circle (shown for illustration only). After the final delay period, a probe appeared at one of the memorandum locations and observers reported the target orientation at this location using a method of adjustment (see Materials and Methods). B, Report variability for each condition. Data are shown as in Figure 2A. Colored lines indicating each observer's performance match colors in Figure 3C. C, Differences in report variability across conditions. Data are shown as in Figure 2B. D, Error distributions and model fit. Green and purple points show crowded and uncrowded conditions, respectively. Data are shown as described in Figure 2C. The model assuming memory performance is independent of cortical spacing (blue line) was again a better fit to the data than the model assuming an influence of cortical spacing, which has been omitted to increase visibility. E, Relationship between critical spacing and memory performance. No correlation between report variability pooled across conditions and critical spacing was found. Solid line indicates regression line of best fit.