Working memory accuracy for multiple targets is driven by reward expectation and stimulus contrast with different time-courses

Abstract

The richness of sensory input dictates that the brain must prioritize and select information for further processing and storage in working memory. Stimulus salience and reward expectations influence this prioritization but their relative contributions and underlying mechanisms are poorly understood. Here we investigate how the quality of working memory for multiple stimuli is determined by priority during encoding and later memory phases. Selective attention could, for instance, act as the primary gating mechanism when stimuli are still visible. Alternatively, observers might still be able to shift priorities across memories during maintenance or retrieval. To distinguish between these possibilities, we investigated how and when reward cues determine working memory accuracy and found that they were only effective during memory encoding. Previously learned, but currently non-predictive, color-reward associations had a similar influence, which gradually weakened without reinforcement. Finally, we show that bottom-up salience, manipulated through varying stimulus contrast, influences memory accuracy during encoding with a fundamentally different time-course than top-down reward cues. While reward-based effects required long stimulus presentation, the influence of contrast was strongest with brief presentations. Our results demonstrate how memory resources are distributed over memory targets and implicates selective attention as a main gating mechanism between sensory and memory systems.

Figure 1

Experimental Design. (a) Working memory paradigm (schematic, in reality gratings were Gabors and background was gray). Trials started with 500 ms of fixation, followed by a Sample phase (3,000 ms in Experiments 1, 2, and 3; 300 ms in Experiments 4 and 5) in which three oriented Gabor patches were displayed (drawn here schematically as square wave gratings), surrounded by colored rings that were used as reward prediction cues in Experiments 1, 2, and 5. Participants remembered the orientations of the Gabors during a 2,000 ms memory maintenance phase and adjusted the orientation of a randomly selected target to match the memorized orientation in the Response phase. Feedback was provided about performance on each trial. (b) In Experiments 1, 2, and 5, the color of the rings around the targets was associated with a specific reward level (valid predictors in Experiments 1 and 5; no predictive value in Experiment 2). The participants first learned the association between color and reward level in a separate color-reward association task (right panel) in which they reported the color of three visual targets as quickly as possible. Feedback was provided about performance after every trial. (c) In Experiments 3 and 4, the contrast of the Gabors was manipulated to modulate the targets’ salience. In Experiments 3 and 4A the three targets differed in contrasts. In Experiment 4B all targets had the same contrast but the contrast level was varied across trials. (d) An overview of the different Experiments and their crucial parameters. Differences among experiments concern the duration of the sample phase, the presence and relevance of colored rings, and the presence and type of contrast manipulations.

Figure 2

Results of Experiment 1. (a) Results of the color-reward association task of Experiment 1 averaged over all participants (n = 10). Correct color reports were generally faster for colors associated with higher rewards. (b) Results of the working memory task of Experiment 1 (n = 10). The error between the reported and the remembered orientation was smaller for stimuli surrounded by a colored ring predicting higher rewards, but only during the sample phase of the task. Error bars represent within-subjects s.e.m.’s. Asterisks indicate significance levels for pairwise comparisons between reward cues (*p < 0.05; **p < 0.01). The open circle indicates a significant difference from the no cue condition (p < 0.05).

Figure 3

Results of Experiment 2. (a) Results of the color-reward association task of Experiment 2 averaged over all participants (n = 8). Correct color reports were generally faster for colors associated with higher rewards. (b) Results of the working memory task of Experiment 2 (n = 8). The error between the reported and the remembered orientation was smaller for stimuli surrounded by a ring in a color that previously predicted higher rewards. This effect was significant when all trials were analyzed together but strongly relied on the first half of the trials and the effect failed to reach significance in the second part of the experiment. Color cues were only presented in the sample phase, or not at all. Error bars represent within-subjects s.e.m.’s. Asterisks indicate significance levels for pairwise comparisons between reward-cues (**p < 0.01). The open circle indicates a significant difference from the no cue condition (p < 0.05).

Figure 4

Results of Experiments 3 and 4. (a) Results of the working memory task of Experiment 3 averaged over all participants (n = 9). With a sample duration of 3,000 ms and either the same medium contrast or different contrasts for the three targets, there was no effect of salience on the precision with which target orientations were reproduced from memory. (b) Results of the working memory task of Experiment 4A averaged over all participants (n = 8). With a sample duration of 300 ms and different contrasts for the three targets, there was a significant effect of salience on the precision with which target orientations were reproduced from memory. (c) Results of the working memory task of Experiment 4B averaged over all participants (n = 8). With a sample duration of 300 ms and the same contrasts for the three targets within a trial but different contrasts between trials, there was again no effect of contrast on the precision with which target orientations were reproduced from memory. Error bars represent within-subjects s.e.m.’s. Asterisks indicate significance levels for pairwise comparisons among target categories (*p < 0.05; **p < 0.01).

Figure 5

Results of Experiment 5. (a) Results of the color-reward association task of Experiment 5 averaged over all participants (n = 8). Correct color reports were generally faster for colors associated with higher rewards. (b) Results of the working memory task of Experiment 5 (n = 8). The error between the reported and the remembered orientation was not affected by colored rings in the Sample phase that predicted the potential rewards. Error bars represent within-subjects s.e.m.’s. Asterisks indicate significance levels for pairwise comparisons among target categories (**p < 0.01).