Dynamic shifts of limited working memory resources in human vision

Abstract

Our ability to remember what we have seen is very limited. Most current views characterize this limit as a fixed number of items-only four objects-that can be held in visual working memory. We show that visual memory capacity is not fixed by the number of objects, but rather is a limited resource that is shared out dynamically between all items in the visual scene. This resource can be shifted flexibly between objects, with allocation biased by selective attention and toward targets of upcoming eye movements. The proportion of resources allocated to each item determines the precision with which it is remembered, a relation that we show is governed by a simple power law, allowing quantitative estimates of resource distribution in a scene.

Fig. 1. Experimental procedure

(A) Stimuli and sequence of events on a location judgement trial. The example shown is a fixation trial with a set size of four items. After the sample display is blanked, subjects' memory for location of a randomly-chosen item is tested by re-displaying the item displaced horizontally through distance Δ. The subject must report the direction of displacement. (B) An orientation judgement trial (this time shown for the saccade condition, with a set size of two items). At the tone, the subject makes a saccade towards the item of a pre-specified color (here green) with the display being blanked during the eye movement. A randomly-chosen item is re-displayed, rotated through an angle Δ, and the subject reports the direction of rotation. Red circles indicate gaze position.

Fig. 2. Performance on the memory task

(A) Proportion of displacements judged outwards from fixation (top row) as a function of the actual displacement (Δ) and number of items in the display (N). Similar plots are shown in the lower row for orientations (rotations) judged clockwise as a function of the actual rotation and number of items. Results from fixation trials are shown in black, saccade trials in red. Curves indicate cumulative gaussian distributions fitted to the response data. Note how the slopes of these functions become flatter with increasing number of items. (B) Precision (determined by the reciprocal of the s.d. of the fitted gaussian) falls as a function of the number of items in the sample display. Error bars indicate ±1 S.E.

Fig. 3. Modelling visual memory performance

(A) The relationship between available memory resources and precision is approximated by a power law (solid blue line; dashed line indicates 95% confidence limits) fitted to the normalized precision values obtained in the first experiment, including both fixation and saccade conditions (circles: location task, triangles: orientation task; black: fixation trials, red: saccade trials; empty symbols: flash cue, filled: no flash cue). (B) Normalized precision as a function of number of items in memory (N) in all conditions. Solid line indicates the prediction of the fitted power-law model. Normalization is with respect to performance with one item (N = 1) in each of the experimental conditions. (C) Response probability as a function of the size of the change (Δ) to the stimulus (i.e., displacement or rotation), for different numbers of items (N), predicted on the basis of the power-law model. σ indicates one standard deviation of the N = 1 response function. Note how the curves become flatter with increasing number of items, corresponding to changes in the gaussian distributions of error in the stored stimulus representation (illustrated in the inset). The dotted vertical line corresponds to a small change to the stimulus, as used in the current study, whereas the dashed vertical line indicates a much larger change. In the latter case performance would be near ceiling for 1–4 items but fall with further increases in set size. (D) Predicted probability correct for stimulus changes of different magnitudes (black lines). σ again indicates one standard deviation of the N = 1 response function. The iso- lines for each multiple of σ were derived directly from C. Red symbols show empirical data from the current study. Green symbols show data from (5). Note how both sets of data are consistent with the power-law model, but different curves arise with differences in the size of stimulus change. See also Fig. S2 for data plotted according to different sizes of change within the current experiment.

Fig. 4. Effects of eye movements and attention

(A) Precision as a function of number of items: memory for saccade targets (filled symbols) and non-targets (empty symbols), when the target is specified endogenously by color (black) or exogenously by a flash (grey). Note the better performance for targets, regardless of the mode of cueing. (B) Memory for an item cued by a flash (filled symbols) and for non-cued items (empty symbols), with no eye movements. (C) Memory for items as a function of fixation order in a sequence of saccades demonstrates how memory for items only one fixation ago is poor compared to the current saccade target. Error bars indicate ±1 S.E.