Neural mechanisms of attending to items in working memory

Abstract

Working memory, the ability to keep recently accessed information available for immediate manipulation, has been proposed to rely on two mechanisms that appear difficult to reconcile: self-sustained neural firing, or the opposite-activity-silent synaptic traces. Here we review and contrast models of these two mechanisms, and then show that both phenomena can co-exist within a unified system in which neurons hold information in both activity and synapses. Rapid plasticity in flexibly-coding neurons allows features to be bound together into objects, with an important emergent property being the focus of attention. One memory item is held by persistent activity in an attended or "focused" state, and is thus remembered better than other items. Other, previously attended items can remain in memory but in the background, encoded in activity-silent synaptic traces. This dual functional architecture provides a unified common mechanism accounting for a diversity of perplexing attention and memory effects that have been hitherto difficult to explain in a single theoretical framework.

Keywords: Attention; Attractor network; Hebbian plasticity; Neural networks; Working memory.

Fig. 1

Conjunctive neurons form a plastic attractor to support attention and working memory. A Two populations of neurons are distinguished based on their inputs. Posterior neurons (green) encode sensory-motor features, whereas prefrontal neurons (blue) are “conjunctive”: i.e. they are able to rapidly increase or decrease their synaptic connectivity with patterns of feature neurons, using a Hebbian associative rule. We simulated 12 feature-selective neurons (f) and 4 freely-conjunctive neurons (c). An active combination of neurons (pink) causes strengthening of synapses in both directions, producing a stable attractor across brain areas. c=conjunctive cells, f=feature cells. W = synaptic weights, i=sensory input. B Sequence of proposed neuronal events during attention, encoding and retrieval in working memory. 1. Sensory input activates features. In this case a vertical red bar located at the top left of the display activates separate feature neurons tuned to orientation, color and location. 2. Features excite conjunctive neurons, which compete. 3. The winning conjunction drives sustained activity. 4. New input to the system (in this case an oblique purple bar at bottom left) disrupts current firing activity, but synaptic weightings remain. 5. Probe feature (in this case red colour) re-activates the original conjunctive unit that encoded the red vertical bar. 6. Conjunctive unit re-activates original features, completing recall.

Fig. 2

Predicting visuospatial WM capacity, encoding and decay. A To simulate WM performance, four objects are presented sequentially, by activating feature neurons (f, activity depicted as a heatmap from dark blue to red) indicating the color, orientation and location of each item. Conjunctive units (c) are shown below as four differently-colored traces. Conjunctive units compete to become active for each object. One conjunctive unit wins for each object, driving activity that persists even after input is removed (yellow parts of heatmap). At the time of the probe, a single feature is stimulated, triggering pattern completion. Recall is accurate if the orientation of the corresponding item is re-activated. Two example trials are shown; note that different patterns of conjunctive units are activated on different trials even for the same stimuli, depending on trial history. Example 1: good encoding. Example 2: weak encoding of the second item. Two conjunctive neurons with similar recent preferences compete to encode object 2 (arrowhead). When it is probed, item 4 is reported instead. B & C When more items are encoded in the model, recall accuracy is reduced, as observed in data (adapted from Luck and Vogel, 1997). D & E The last item encoded in the model is recalled better than others, as it remains active in the focus of attention during the delay period, matching observed serial order curves. Figure adapted from (Gorgoraptis et al., 2011) indicates the probability of reporting the target item as calculated by fitting the distribution of responses in a similar task. F&G Shorter encoding durations reduce modelled recall accuracy. Data from a similar task (adapted from Bays et al., 2011b) where adding items reduced both initial encoding rate and asymptote. The model qualitatively reproduces the interaction observed in human performance. H & I The model predicts faster memory decay when more items are stored. This matches the empirical interaction between memory-set size and delay. Data adapted from (Pertzov et al., 2016) shows the modelled probability of reporting the target. Note that at very short delays, model recall was more accurate than in human data.

Fig. 3

Shifting the focus of attention in WM. A Experiment (Zokaei et al., 2014a) where participants remembered two items, each comprising three features: color, location and orientation. During the retention interval, a color was shown, and as a secondary task, the location of the corresponding object had to be recalled. At the end of the delay, a color was shown which could indicate the same (“congruent”) or different (“incongruent”) object than the one tested during the delay. Participants then reported the orientation of the corresponding object. Reproduced under the terms of the Creative Commons Attribution 3.0 Unported (CC BY 3.0) license (https://creativecommons.org/licenses /by/3.0) from Fig. 1A of Zokaei et al. (2014a), The Journal of Neuroscience. January 1, 2014. 34(1);158–162. B Similar events were simulated, with an incidental cue (IC) during the delay. If the first object was cued, then persistent delay activity shifted to the cued item. C&D The model predicts that the item in the focus of attention before recall is reported more accurately, matching data. Probability of target from mixture model fitted to data of Zokaei et al. 2014. E Decoding direction of object 2 from feature-selective units during the delay, on trials where the first item was cued (IC). Decodability is low but still above chance after the cue, with below-chance performance of a cross-decoder trained on trials where the second item was cued (full analysis Fig. S12).

Fig. 4

Introducing a pulse of excitation during the delay period. A After presenting two items, during the delay all feature neurons f received an excitatory input pulse i=+1, consequently activating conjunction neurons. B&C We tried decoding the identity of each of the two stimuli from feature neuron activity. Although the first object was not decodable without the pulse, it became transiently distinguishable (*) after the pulse. This matches the observed increase in decodability after TMS (Rose et al., 2016). D&E Stronger pulses altered model performance, abolishing the benefit for the second item, which was in the focus of attention. The pulse disrupted persistent activity, re-instating competition between conjunctive neurons. This results in randomly re-selecting which of the stable states of the plastic attractor is active. The prediction matches observed effects of TMS targeting motion-selective cortex (probability of selecting the target in mixture model fitted to data from (Zokaei et al., 2014a).

Fig. 5

Conjunctive unit representations are stable over short timescales. Conjunctive units change their selectivity over short periods. If selectivity were stable, neural patterns should be similar when the stimulus is the same. We compared similarity of the pattern of an earlier trial, to trial n, during the delay periods of a series of 1-item trials. A) The similarity of the conjunctive neurons’ delay activity pattern is calculated for trials where the stimuli were identical (blue line) or different (red line). Patterns were more similar when stimuli were the same, compared to when stimuli were different, indicating “classical encoding” at least for nearby trials. This classical behavior decreased with the temporal distance between trials. Since we modelled the extreme case where neurons are purely conjunctive, with no feature selectivity, consistency of pattern is completely abolished after about 6 trials. B-D) The model predicts that interference reduces pattern similarity over time by overwriting the synaptic weights. If the objects in intervening trials share one feature with the nth trial object, but mismatch on the other feature dimension, then we say the conjunction between the two feature dimensions is “violated”. B) When the intervening trial contained a violation, the patterns on the n-2 and nth trials reflected the stimuli much more weakly, indicating interference or overwriting of the original conjunction. C and D) Trials 3-back and 4-back were similarly examined, this time asking how many intervening conjunction violations occurred. The more overwriting that occurred between the n-3 and nth trials, the less classical encoding could be observed.