The time course of encoding and maintenance of task-relevant versus irrelevant object features in working memory

Abstract

Access to WM can be restricted on the basis of goal-relevant properties such as spatial location. However, the extent of voluntary control over which features of an attended multi-feature object are encoded and maintained in WM is debated. Some evidence suggests that attending to an object leads to obligatory storage of all of its features, whereas other evidence suggests that access to WM can be restricted to only goal-relevant features. Another possibility is that all features are initially encoded, but irrelevant features are removed from WM over time. To address these various possibilities, we used pattern classification of EEG signals to track the temporal evolution of representations reflecting the encoding and storage of task-relevant and irrelevant features in WM. In different blocks, participants remembered the orientation, color or both orientation and color of a colored, oriented grating. The color and orientation of the grating was randomly drawn from two distinct feature bins on each trial. To examine trial-specific activity reflecting storage of the object's features, a support vector machine (SVM) classifier was trained to classify what bin the stimulus features came from. Importantly, for orientation, the classifier produced reliably above-chance classification across the delay when orientation was task-relevant but not when it was task-irrelevant. Interestingly, orientation could be accurately classified on trials for which both orientation and color were remembered. Moreover, a separate measure corresponding to the probability of a feature belonging to the correct bin was significantly higher when orientation was task-relevant compared to task-irrelevant during encoding. Above-chance classification for color was only present during the initial 500 msec across all conditions. Our results suggest that although information about all of an object's features is present in the initial stimulus-evoked neural response, information about the task-irrelevant features is attenuated during stimulus encoding and is largely absent throughout the delay.

Keywords: Feature binding; Support-vector machine classifier; Top-down control; Working memory.

Figure 1.

Behavioral task trial sequence (stimuli not depicted to scale). Participants were presented with a two-feature object in the center of the screen, and, in different blocks, were asked to remember its orientation, its color or both color and orientation. Each trial started with the presentation of a fixation cross, followed by a 200 ms presentation of the sample display. The sample was followed by 1800 ms long blank delay interval during which participants were asked to remember the sample stimulus. At test, participants were asked report whether a change in the task-relevant feature had occurred by pressing a key.

Figure 2.

Classifier accuracy for task-relevant feature dimension decoding. (A) Time-series decoding of task-relevant feature dimension. Classifier accuracy was consistently above chance throughout the whole trial interval. Thick colored bar at the top of the plot depicts significant time points (corrected for multiple comparisons using permutation testing). Gray bar at the bottom of the plot corresponds to stimulus presentation. Dashed line represents empirically derived chance. Shaded areas correspond to ±1 SEM. (B) Bar-plot of classifier accuracy across three trial intervals: pre-stimulus (–200–0 ms), encoding (0–300 ms) and delay (300–2000 ms). Classifier accuracy was significantly above chance during the entire trial interval including the pre-stimulus interval; however, there was a significant increase in accuracy following stimulus presentation and during the delay interval. Error bars correspond to ±1 SEM. (C) Time-frequency plot of classifier accuracy. Above-chance classifier accuracy was supported by patterns of oscillatory activity across a wide range of frequencies (all time-frequency points significant p < .001; corrected for multiple comparisons using permutation testing). Activity in the lower frequency range (4–15Hz) mainly supported decoding accuracy time-locked to the presentation of the stimulus, whereas higher frequencies (>15 Hz) supported decoding at the level of experimental blocks.

Figure 3.

Classifier accuracy for feature identity decoding from evoked power (feature bin 1 vs. feature bin 2). Classification of orientation and color is depicted in the left (ABC) and right (DEF) panels, respectively. (A, D) Classifier accuracy for feature identity classification across time. Feature identity was decodable during the initial ~700 ms of the trial irrespective of feature relevance. Representation of the task-relevant feature persisted the longest. Thick colored bars at the top of the plots depict significant time points (corrected for multiple comparisons using permutation testing). Gray bars at the bottom of the plots correspond to stimulus presentation. Dashed lines represent empirically derived chance. Shaded areas correspond to ±1 SEM. (B, E) Bar-plot of classifier accuracy across three trial intervals: pre-stimulus (–200–0 ms), encoding (0–300 ms) and delay (300–2000 ms). Classifier accuracy was higher during encoding compared to both pre-stimulus and delay intervals. There was no significant interaction between time interval and feature relevance. Error bars correspond to ±1 SEM. (C, F) Time-frequency decoding of feature identity. Above chance classification of feature identity was supported by a wide range of frequencies (orientation 4–25 Hz, color 4–30 Hz). Time-frequency points with p > .001 are masked with semi-transparent white color (corrected for multiple comparisons using permutation testing).

Figure 4.

Classifier accuracy for feature identity decoding from total power (feature bin 1 vs. feature bin 2). Classification of orientation and color is depicted in the left (ABC) and right (DEF) panels respectively. (A, D) Classifier accuracy for feature identity classification across time. Classification of orientation was reliably above chance only when orientation corresponded to the task-relevant feature (orientation relevant and both relevant conditions). Decoding of orientation identity in orientation-irrelevant condition was only successful during the initial ~500 ms. Classification of color was only reliable during the initial ~500 ms irrespective of color relevance. Thick colored bars at the top of the plots depict significant time points (corrected for multiple comparisons using permutation testing). Gray bars at the bottom of the plots correspond to stimulus presentation. Dashed lines represent empirically derived chance. Shaded areas correspond to ±1 SEM. (B, E) Bar-plot of classifier accuracy across three trial intervals: pre-stimulus (–200–0 ms), encoding (0–300 ms) and delay (300–2000 ms). There was a significant interaction between interval and feature relevance for orientation. Classifier accuracy was higher during encoding and delay intervals when orientation was task-relevant compared to task-irrelevant. There was no significant interaction between interval and feature relevance for color. Error bars correspond to ±1 SEM. (C, F) Time-frequency decoding of feature identity. Above chance classification of orientation identity was supported by patterns of oscillatory power in the alpha-band range (8–14 Hz) across the delay interval. Time-frequency points with p > .001 are masked with semi-transparent white color (corrected for multiple comparisons using permutation testing).

Figure 5.

Comparison of classifier evidence between different conditions. Classifier evidence for orientation and color is depicted in the left (AB) and right (CD) panels respectively. Gray bars at the bottom of the plots correspond to stimulus presentation. Thick colored bars at the top of the plots depict time points with significant differences between conditions (corrected for multiple comparisons using Benjamini-Hochberg correction). Dashed lines represent P = 0.5. Shaded areas correspond to ±1 SEM. (A, B) Classifier evidence calculated from evoked power. There was a significant difference in classifier evidence between orientation relevant and orientation irrelevant conditions during the initial 100–600 ms (black thick line at the top of the plot). There was also a significant difference in classifier evidence between conditions where orientation was the only task relevant feature and where orientation was remembered together with color (gray bar at the top of the plot). No significant differences between conditions were observed for color. (C, D) Classifier evidence calculated from total power. Similar to evoked power classifier evidence, there was a significant difference between orientation relevant and orientation irrelevant conditions during the initial encoding of the stimulus. No significant differences between conditions were observed for color.