Temporal dynamics of encoding, storage, and reallocation of visual working memory

Abstract

The process of encoding a visual scene into working memory has previously been studied using binary measures of recall. Here, we examine the temporal evolution of memory resolution, based on observers' ability to reproduce the orientations of objects presented in brief, masked displays. Recall precision was accurately described by the interaction of two independent constraints: an encoding limit that determines the maximum rate at which information can be transferred into memory and a separate storage limit that determines the maximum fidelity with which information can be maintained. Recall variability decreased incrementally with time, consistent with a parallel encoding process in which visual information from multiple objects accumulates simultaneously in working memory. No evidence was observed for a limit on the number of items stored. Cuing one display item with a brief flash led to rapid development of a recall advantage for that item. This advantage was short-lived if the cue was simply a salient visual event but was maintained if it indicated an object of particular relevance to the task. These cuing effects were observed even for items that had already been encoded into memory, indicating that limited memory resources can be rapidly reallocated to prioritize salient or goal-relevant information.

Figure 1. Assessing effects of exposure duration on recall of orientation

(a) The recall task used in Exp 1. An array of colored oriented bars was presented for a variable exposure duration, followed by a pattern mask. After a blank retention interval, a probe bar appeared and subjects used a response dial to adjust its orientation to match the item with the same color in the memory array (the target). The angular difference between response and target orientations was taken as a measure of recall error. (b) Three hypotheses regarding the evolution of recall precision with exposure time, as a function of the number of items in the memory array. Here, lighter shades indicate more items stored in memory. The top panel is based on an assumption of limited storage capacity: as more items are stored the maximum-attainable precision declines. The middle panel depicts the case of limited encoding capacity: eventually the same level of precision is reached regardless of array size. Finally, the lower panel shows expected performance when both storage and encoding capacities are limited. Compare these possible results with the actual findings in Fig 2a.

Figure 2. Temporal evolution of working memory precision

(a) Recall precision as a function of exposure duration and number of array items (N). Precision is defined as the reciprocal of the standard deviation of error in the reproduction task. Error bars indicate ± 1 SE. Dashed lines indicate RC curves (see Methods) with parameters that best fit the temporal evolution of recall precision at each array size. (b) The initial rate of encoding into memory was estimated from the rate parameter of the fitted RC curve (inset). The initial rate (black symbols) declined with array size according to a simple inverse relationship (i.e. rate proportional to 1/N, red curve). (c) The upper lmiit on precision was estimated from the capacity parameter of each RC curve (inset). This storage limit (black symbols) declined with array size more slowly than encoding rate, following a power law relationship (i.e. maximum precision proportional to N^−λ, green curve).

Figure 3. Components of error in the working memory task

(a) The distribution of responses was decomposed into a mixture of three separate components: responses distributed with Gaussian variability around the correct (target, T) orientation (top), responses distributed around the orientations of other, non-target (NT) items in the memory array (middle), and random responses distributed uniformly throughout the response space (bottom). (b) Maximum likelihood estimates of parameters of the mixture model illustrated in (a), for different array sizes and exposure durations. Gaussian variability in target responses (black symbols) increases with array size but decreases with exposure duration. The frequencies of non-target (blue) and random (red) responses also increase with array size. Note that random responding declines rapidly with increasing exposure duration, but non-target (misbinding) errors maintain a constant frequency at intermediate and long durations. Error bars indicate ± 1 SE.

Figure 4. Assessing cueing effects on recall performance

(a) The recall task used in Exp 2. Memory arrays consisted of two randomly-oriented bars (one red, one blue). A randomly-selected bar was cued by a briefly flashed white disk presented simultaneously with the onset of the memory array. After a variable post-cue display period, in which the memory array remained visible, a pattern mask was presented. Subjects adjusted a probe bar to reproduce the orientation of one of the array items, as in Exp 1. (b) The recall task used in Exp 3. The procedure was identical to Exp 2, except that the memory array was displayed for 1000 ms before one of the bars was cued, in addition to the variable post-cue display period. In Exps 2A & 3A, the cue predicted which item would be probed on 2/3 trials; in Exps 2B & 3B, the cue was not predictive of the probe.

Figure 5. Time course of cueing effects on encoding and maintenance

(a) Recall precision as a function of exposure duration when a predictive cue was presented at array onset, for trials on which memory was probed for the cued item (valid trials, black) and non-cued item (invalid trials, red). Error bars indicate ± 1 SE. Asterisks indicate time points at which there was a significant recall advantage for cued over non-cued items (p < 0.05). Dotted vertical line and shaded area indicate onset and duration of the cue event. (b) Recall precision plotted as in (a), but for subjects presented with non-predictive (task-irrelevant) cues. Note that, unlike for predictive cueing, the advantage for the cued item is abolished at longer exposure durations. (c) Recall precision for predictive cues presented after 1000 ms exposure to the memory array, plotted as a function of post-cue display time. Note that high-resolution representations of both items are already stored in memory at the time of cue presentation (0 ms condition). (□) indicates a cued-item advantage with borderline statistical significance (p = 0.054). (d) Recall precision for non-predictive cues after 1000 ms exposure. Note the cued item advantage is again abolished at longer exposures.

Figure 6. Recall of cued and non-cued items relative to baseline

(a) Recall precision in Exp 4 on invalid- and valid-cue trials, in which one array item was cued with a flash, and on neutral-cue trials, in which both items were flashed. Cues were presented after 1000 ms, and post-cue exposure was 400 ms. Single cues were predictive of which item would be probed (valid trials twice as frequent as invalid). Asterisks indicate significant differences, p < 0.05. (b) Maximum likelihood estimates of parameters of the mixture model (illustrated in Fig 3), for invalid-, valid- and neutral-cue trials. Note cueing condition is reflected in variability of the Gaussian response component, while non-target and uniform components make negligible contribution to errors.