Robust representation of stable object values in the oculomotor Basal Ganglia

Abstract

Our gaze tends to be directed to objects previously associated with rewards. Such object values change flexibly or remain stable. Here we present evidence that the monkey substantia nigra pars reticulata (SNr) in the basal ganglia represents stable, rather than flexible, object values. After across-day learning of object-reward association, SNr neurons gradually showed a response bias to surprisingly many visual objects: inhibition to high-valued objects and excitation to low-valued objects. Many of these neurons were shown to project to the ipsilateral superior colliculus. This neuronal bias remained intact even after >100 d without further learning. In parallel with the neuronal bias, the monkeys tended to look at high-valued objects. The neuronal and behavioral biases were present even if no value was associated during testing. These results suggest that SNr neurons bias the gaze toward objects that were consistently associated with high values in one's history.

Figure 1.

Procedure for learning stable object values. A, Stable object–reward association task. Among a set of eight fractal objects, four were assigned as “high-valued” objects and the other four were assigned as “low-valued” objects. On each trial, one of the eight objects was presented at one of four positions, and the monkey made a saccade to it. If the object was a high-valued object, a large reward was delivered. If the object was a low-valued object, a small reward was delivered. For each set of visual objects, the learning procedure was done repeatedly across many daily sessions, during which each fractal remained to be either a high-valued object or a low-valued object. B, Well learned fractals (≥5 learning sessions) used for monkey G. Each row of eight fractals (shown on each of left and right sides) were used as a set of objects. Among them, the left four fractals were high-valued objects (associated with a large reward) and the right four fractals were low-valued objects (associated with a small reward). Monkey G learned 648 fractals, among which 288 (shown here) were well learned. Monkey D learned 696 fractals, among which 280 were well learned. Many of the fractals were shared by both monkeys.

Figure 2.

Procedures for testing the stable object value coding. A, Procedure for testing SNr neurons' responses to high-valued objects and to low-valued objects. While the monkey was fixating a central spot of light, 1–4 fractal objects (pseudorandomly chosen from a set of 8 objects) were presented sequentially in the neuron's preferred location. The monkey was rewarded 300 ms after the final object disappeared. The reward was thus not associated with particular objects. B, Free-viewing task for testing saccade behavior. In one session, a set of eight fractal objects were presented and the monkey was free to look at these objects (or somewhere else) by making saccades between them. For one presentation, four objects were chosen randomly from the eight objects and for the next presentation the other four objects were chosen, and this cycle was repeated. No reward was delivered during the free viewing. Occasionally, a white small dot was presented at one of four positions. If the monkey made a saccade to it, a reward was delivered to the monkey. The testing procedures were done on separate days from the learning procedure (Fig. 1) to avoid possible influences of working memory or short-term memory.

Figure 3.

Object-selective responses of an SNr neuron to novel fractal objects. A, Antidromic activation of an SNr neuron to electrical stimulation of the ipsilateral SC. The antidromic nature of the spikes was confirmed by a collision test (right, bottom). B, Responses of the SNr neuron to novel fractal objects, shown by rasters of dots (top) and spike density functions (bottom). The objects were presented at the neuron's preferred location while the monkey was fixating at the central fixation point (see Fig. 2A). The same neuron as shown in Figure 10. Top and bottom four objects were assigned as high-valued and low-valued objects in the later learning.

Figure 4.

Stable object value coding of an SC-projecting SNr neuron. A, Learning schedule for a set of fractal objects used to test the responses of the SNr neuron (as shown in B). Monkey D learned the set of objects for 13 sessions across days (black bars). Five days after the last learning session, the SNr neuron was recorded (red bar). One learning session was done in 1 d. B, The responses of the SNr neuron to the learned object set: high-valued objects (top) and low-valued objects (bottom). For each fractal, the activity is aligned on the onset of the fractal (time 0). The horizontal bar shown above each raster plots indicate the duration of object presentation (0–400 ms).

Figure 5.

An SC-projecting SNr neuron encoding stable values of many objects. We tested the neuron's responses to 120 well learned fractal objects (≥5 learning sessions) (left). Shown superimposed on right are the neuron's responses to 60 high-valued objects (red) and 60 low-valued objects (blue). Firing rates (shown by spike density functions) are aligned on the onset of the object (time 0). The object disappeared at 400 ms. The number of learning sessions for these objects ranged from 5 to 22. The last learning of object–reward association had been done 3 d before the neuronal recording.

Figure 6.

Actual experimental schedule for monkey G. The monkey experienced 712 fractal objects grouped into 89 sets (8 fractals each). Each row indicates the schedule for each object set (black, learning; red, testing of neuronal activity) across the days of the monkey's career in this study. Among the 89 object sets, 81 sets were used for object–reward association learning; eight sets were used only for testing. As the monkey learned many fractal objects, new object sets were added for object–reward association. Thus, at any point during the career, the monkey had learned many object sets in different degrees (i.e., different number of learning sessions). This means that the neuronal learning curve can be inferred from weak to strong response biases of one SNr neuron (recorded on a particular day) to the object sets with small to large amount of learning, as illustrated in Figure 10A. Many learned object sets were omitted from learning sessions and, after many days (up to 165 d) of no further learning, were used for testing SNr neurons' responses and the monkey's saccade behavior. The shaded area indicates the experiments shown in Figure 5.

Figure 7.

Stable object value coding of SNr neurons in monkey D (left) and G (right). They are shown separately for neurons antidromically activated from the SC (left) (D: n = 52; G: n = 36) and neurons not antidromically activated from the SC (right) (D: n = 33; G: n = 30). A–D, Averaged responses to high-valued objects (red) and low-valued objects (blue). The data were obtained for well learned sets (≥5 learning sessions). E–H, Comparison between the responses to high-valued objects and the responses to low-valued objects. Plotted for each recording session (each dot) are the average response to high-valued objects (abscissa) and the average response to low-valued objects (ordinate). The response was measured as the difference in firing rate between the test window (100–400 ms after object onset) and the baseline window (500–1000 ms before fixation point onset). I–L, Discrimination between high-valued and low-valued objects in individual recording sessions measured by ROC area based on spike counts in a test window. ROC 0.5 would mean no differentiation; ROC 1.0 would mean that the neuronal activity was always lower (or more inhibited) by high-valued objects than low-valued objects; ROC 0.0 would mean the opposite. Black bars indicate neurons with statistically significant discrimination assessed by Wilcoxon rank-sum test (p < 0.05). A triangle indicates the mean of the ROC areas: 0.79 for I and J, 0.73 for K, and 0.76 for L′.

Figure 8.

An SC-projecting SNr neuron retaining object values after a long period of no learning. A, Learning schedule for a set of fractal objects. Monkey D learned the set of objects for 14 sessions across days (black bars). Then the learning for this object set was stopped for 108 d, until the SNr neuron was tested (red bar). The same format as Figure 4A. B, After the long retention period, the SNr neuron clearly discriminated between high-valued and low-valued objects. Note that during the retention period, the monkey learned many other fractals.

Figure 9.

Long-term retention of object values by SNr neurons. Comparison between short retention periods (<20 d, left) and long retention periods (>100 d, right). The data were obtained from 151 and 22 SNr neurons for short and long retention periods, respectively, in two monkeys for well learned sets (≥5 learning sessions). The same format as in Figure 7. A, B, Averaged responses to high-valued objects (red) and low-valued objects (blue). C, D, Comparison between the responses to high-valued objects and the responses to low-valued objects. E, F, Discrimination between high-valued and low-valued objects in individual recording sessions measured by ROC area. Black bars indicate neurons with statistically significant discrimination assessed by Wilcoxon rank-sum test (p < 0.05). A triangle indicates the mean of the ROC areas: 0.77 for short retention periods and 0.74 for long retention periods. There was no statistical difference in the mean ROC area between the two groups of data (t test, p = 0.18). Data in C–F are based on each recording session (short retention period, n = 563; long retention period, n = 76).

Figure 10.

SNr neurons learn stable object values gradually. A, Learning schedule for 19 sets of fractal objects used for testing the responses of an SNr neuron (as shown in B and C). Note that multiple sets of fractals were added for learning at different points in the monkey's career, so that when an SNr neuron was recorded, the amount of learning varied across the fractal objects. B, The responses of an SNr neuron to fractal object sets with seven different degrees of learning (0–19 sessions). The responses are shown for each set, separately for high-valued objects (red) and low-valued objects (blue). C, Learning curve of the neuron shown in B. The neuron's discrimination between high-valued and low-valued objects for each object set (expressed as an ROC area) is plotted against the number of learning sessions for the object set. Data points are connected by lines to illustrate the neuron's learning curve. For sessions 0 and 5, we used two object sets, and the mean ROC area is used for the learning curve. D, Averaged learning curve across 118 SNr neurons in two monkeys. Data point on right indicates the average of neuronal biases that were obtained >100 d after the last learning. Error bars, ±1 SEM.

Figure 11.

Learning curves for individual SNr neurons in monkey D (left) and monkey G (right). A, B, For each neuron, its discrimination between high-valued and low-valued objects (expressed as an ROC area) is plotted against the number of learning sessions for all sets of fractals tested, and the data points thus obtained are connected with lines in a unique color (which reflects the neuron's learning curve). An example is shown in Figure 10C. C, D, Distribution of the linear correlation coefficients between the ROC area and the number of learning sessions obtained for individual neurons. The distribution was shifted toward positive values, indicating that each neuron's discrimination between high-valued and low-valued objects became stronger with object–reward association learning. The mean correlation coefficients were 0.48 for monkey D and 0.41 for monkey G (triangles). The neuronal data obtained by <3 sets were excluded.

Figure 12.

Monkeys learned stable object values gradually. A, Example behavioral learning curve. The monkey's preference among a set of eight objects (shown above) is plotted against the number of learning sessions. The monkey's preference was tested in a free-viewing task (Fig. 2B). The testing was done at least 1 d after the last learning. Inset graph shows, as an example, the total numbers of saccades to four high-valued objects (red) and four low-valued objects (blue) in the free-viewing task. B, The averaged behavioral learning curve across 128 sets of fractals in two monkeys. Data point on right indicates the average of behavior biases obtained >100 d after the last learning. Error bars, ±1 SEM.

Figure 13.

Weak influences of flexibly changing object values on activity of an SNr neuron. A, Flexible object–reward association task. The monkey was required to hold gaze on the fixation spot until it was turned off (overlap period, 400 ms) and then make a saccade to the object. Two fractal objects were associated with large and small rewards in a reversible manner in blocks of trials (1 block, 30 trials). On most trials (4 of 5 trials) one of the two objects was presented at the neuron's preferred position (forced trials). Occasionally (1 of 5 trials) two objects were presented and the monkey had to choose one of the objects (choice trials, not shown). B, Changes in the choice of monkey D on choice trials across four blocks. The plots close to the horizontal red bars indicate choices preferring the high-valued object (i.e., whichever object was recently associated with the large reward). C, Spike activity of a single SC-projecting SNr neuron during the flexible object–reward association task aligned on the onset of the object (time 0). Vertical red and blue bars on the left of the rasters indicate large-reward and small-reward trials. The purple tick in each raster line indicates the onset of the saccade toward the presented object. The averaged responses are shown separately for the high-valued object (red) and the low-valued object (blue). Black horizontal bars indicate the overlap period. The data were obtained from the same neuron as shown in Figure 5.

Figure 14.

Flexible object value coding of SNr neurons shown separately for two monkeys. A, B, Average responses of SNr neurons to the high-valued object (i.e., recently associated with a large reward) (red) and the low-valued object (i.e., recently associated with a small reward) (blue). The data were obtained from 39 neurons in monkey D (left) and 37 neurons in monkey G (right). C, E, SNr neurons' discriminations between high-valued and low-valued objects measured by ROC area based on spike counts in a test window. Black bars indicate neurons with statistically significant discrimination assessed by Wilcoxon rank-sum test (p < 0.05). A triangle indicates the mean of the ROC areas (monkey D: 0.58; monkey G: 0.54). D, F, The mean saccade reaction times (Saccade RT) for large-reward trials were significantly shorter for high-valued than for low-valued objects (paired t test, p ≈ 0 in both monkeys). Note that the reaction time was measured from the offset of the fixation point. For (A–F), the neuron's responses in the initial four trials in each block have been excluded. G, H, Behavioral (black) and neuronal (red) learning curves associated with flexible object values. The behavioral learning curve is the choice rate for the high-valued object. The neuronal learning curve is the averaged ROC areas across trials after the reversal of the object–reward contingency.