Separating memoranda in depth increases visual working memory performance

Abstract

Visual working memory is the mechanism supporting the continued maintenance of information after sensory inputs are removed. Although the capacity of visual working memory is limited, memoranda that are spaced farther apart on a 2-D display are easier to remember, potentially because neural representations are more distinct within retinotopically organized areas of visual cortex during memory encoding, maintenance, or retrieval. The impact on memory of spatial separability in depth is less clear, even though depth information is essential to guiding interactions with objects in the environment. On one account, separating memoranda in depth may facilitate performance if interference between items is reduced. However, depth information must be inferred indirectly from the 2-D retinal image, and less is known about how visual cortex represents depth. Thus, an alternative possibility is that separation in depth does not attenuate between-items interference; it may even impair performance, as attention must be distributed across a larger volume of 3-D space. We tested these alternatives using a stereo display while participants remembered the colors of stimuli presented either near or far in the 2-D plane or in depth. Increasing separation in-plane and in depth both enhanced performance. Furthermore, participants who were better able to utilize stereo depth cues showed larger benefits when memoranda were separated in depth, particularly for large memory arrays. The observation that spatial separation in the inferred 3-D structure of the environment improves memory performance, as is the case in 2-D environments, suggests that separating memoranda in depth might reduce neural competition by utilizing cortically separable resources.

Figure 1

Each trial started with a 500-ms fixation period during which only the 16 placeholders were shown. Here, light and dark circles indicate placeholders on the far and near depth planes, respectively (this is only for visualization purposes—all placeholders were the same shade of gray in the actual experiment). Next, two memory targets were presented for 150 ms, followed by a 750-ms delay. After the delay, a color wheel was presented together with a cue outlining one of the previous target locations, and participants moved the cursor to report the hue previously shown at the cued location. The two target colors were presented in either the same or different depth planes in 3-D coordinates (same vs. different) and either close or far in 2-D space (see insert at top right). The lower left insert shows the color wheel that we used in the experiment.

Figure 2

Results of Experiment 1 as a histogram of the responses centered around the target color, shown collapsed across all participants and conditions. The nontarget colors were aligned to approximately −90° (±10°) relative to the target color by flipping the sign of responses on trials where the nontarget was +90° (±10°) relative to the target (note that the width of the shaded green area reflects the ±10° jitter in the uncued target color). Swap errors are apparent from the small bump centered on the nontarget color.

Figure 3

Results of Experiment 1 in terms of the parameters from mixture modeling. (A) The standard deviations are lower when two memory items are spatially far away or when they are on different depth planes (lower standard deviation is associated with higher precision). *p < 0.05. (B) There are systematic biases away from the nontarget color in all conditions but no significant differences in biases between conditions. (C) There are no significant differences in swap error rate, nor in (D) guess rate. (E) Four kernel density plots of group-level error responses of each condition centered around the target color (from left: same-close, different-close, same-far, and different-far). The shapes of the distributions qualitatively agree with the parameters from the model. Error bars (in A, B, and C,) represent ±1 standard error of the mean.

Figure 4

Experimental procedure for Experiment 2. (A) In this single-probe change-detection paradigm, each trial started with the presentation of 12 placeholders. Placeholders could have one of three possible depth relationships: all on the near depth plane, all on the far depth plane, or half on the near and the other half on the far depth plane. After 500 ms, two, four, six, eight, or 12 colored memory items were presented for 500 ms, followed by a 900-ms delay period. Next, a single test item was presented at a location previously occupied by one of the memory items, and participants indicated whether the color of the test was the same as or different from the color of the memory target previously shown at that location. (B) The independent depth-discrimination task. On each trial, two placeholders briefly appeared, each on a different depth plane. Participants indicated whether the target (in green) was on the near or far plane. Performance on this task was used as an indicator of how well participants could perceive depth using our stereo-display setup.

Figure 5

Main results of Experiment 2. (A) Visual-working-memory capacity (Cowan's k) as a function of set size. There were no differences in capacity when memory items were displayed on planes at the same (red) or different (blue) depths. Observed changes in k as a function of set size are consistent with previous studies (Cowan & Morey, 2006). (B) The impact of depth separation (on the y-axis) was calculated by taking the capacity k for items presented on different depth planes minus the k for items presented on the same depth plane. Thus, larger numbers indicate a larger benefit of presenting items separated in depth. The ability of participants to discriminate the two depth planes in our experimental setup (on the x-axis) was positively correlated with the benefits they gained from items presented on different depth planes. Shaded regions indicate ±1 standard error of the mean.

Figure 6

The degree of positive correlation between depth-discrimination ability (on the x-axis) and performance on the visual-working-memory task (on the y-axis). Participants who performed better on the depth-discrimination task also performed better on the visual-working-memory task at larger set sizes, but only when the memoranda were on different depth planes (upper row). There was no correlation between performance on the depth-discrimination task and on the visual-working-memory task when the memoranda were in the same depth plane (middle row). The benefit associated with having the memoranda separated into different depth planes (difference in k value on the y-axis) grew stronger as set size increased (bottom row in panels).

Figure 7

Participants who exhibited better depth discrimination (upper panel), based on a median split of performance in the independent depth-discrimination task, benefited more from the presence of depth information, particularly at high set sizes. **p < 0.01. The error bars represent ±1 standard error of the mean. For participants who exhibited worse depth discrimination (lower graph), the k value appeared to be lower when memoranda were on different depth planes, but this did not reach significance. Note that the performance from both groups was comparable when the memoranda were on the same depth plane (compare red lines between the two panels).