Impact of expected value on neural activity in rat substantia nigra pars reticulata

Abstract

The substantia nigra pars reticulata (SNr) is thought to serve as the output of the basal ganglia, whereby associative information from striatum influences behavior via disinhibition of downstream motor areas to motivate behavior. Unfortunately, few studies have examined activity in SNr in rats making decisions based on the value of predicted reward similar to those conducted in primates. To fill this void, we recorded from single neurons in SNr while rats performed a choice task in which different odor cues indicated what reward was available on the left or on the right. The value of reward associated with a leftward or rightward movement was manipulated by varying the size of and delay to reward in separate blocks of trials. Rats were faster or slower depending on whether the expected reward value was high or low, respectively. The number of neurons that increased firing during performance of the task outnumbered those that decreased firing. Both increases and decreases were modulated by expected value and response direction. Neurons that fired more or less strongly for larger reward tended to fire, respectively, more or less strongly for immediate reward, reflecting their common motivational output. Finally, value selectivity was present prior to presentation of cues indicating the nature of the upcoming behavioral response for both increasing- and decreasing-type neurons, reflecting the internal bias or preparatory set of the rat. These results emphasize the importance of increasing-type neurons on behavioral output when animals are making decisions based on predicted reward value.

Figure 1. Task and behavior

A. Choice task during which we varied the delay preceding reward delivery and the size of reward. Figure shows sequence of events in each trial in 4 blocks in which we manipulated the time to reward or the size of reward. Trials were signaled by illumination of the panel lights inside the box. When these lights were on, nosepoke into the odor port resulted in delivery of the odor cue to a small hemicylinder located behind this opening. One of three different odors was delivered to the port on each trial, in a pseudorandom order. At odor offset, the rat had 3 seconds to make a response at one of the two fluid wells located below the port. One odor instructed the rat to go to the left to get reward, a second odor instructed the rat to go to the right to get reward, and a third odor indicated that the rat could obtain reward at either well. One well was randomly designated as short and the other long at the start of the session (block 1). In the second block of trials these contingencies were switched (block 2). In later blocks (3–4) we held the delay preceding reward delivery constant while manipulating the size of the expected reward. B. The height of each bar indicates the percent choice of short delay versus long delay (top) and big reward versus small reward (bottom) taken over all free-choice trials. C–D. The height of each bar indicates the percent correct (C) and reaction time (D) on forced-choice trials across all recording sessions. E. Boxes indicate the approximate extent of recording sites where SNr neurons were collected for each animal. Consistent with previous reports in SNr, baseline firing rates were high (mean = 35 spikes/second; SD = 20) and waveform durations were short (mean = 0.86 ms; SD = 0.47) (Gulley et al., 1999; Gulley et al., 2002b; Meyer-Luehmann et al., 2002; Deransart et al., 2003). Asterisks: planned comparisons revealing statistically significant differences (t-test, p < 0.05). Error bars indicate standard errors.

Figure 2. Activity of single neurons in SNr reflects an interaction between expected value and direction

A. Activity of a single SNr neuron averaged over all trials for each condition aligned on odor port exit during all 8 conditions (4 rewards × 2 directions). Histogram represents average activity over the last ten trials (after learning) for each condition in a block of trials. Each tick mark is an action potential and trials are represented by rows. All trials are shown. B. Results of a two-factor anova with value and direction as factors (p < 0.05). Firing rate was taken from odor onset to port exit. The height of each bar indicates the percentage of neurons that exhibited a main effect of value with no interaction effect, a main effect of direction with no interaction effect or an interaction between the value and direction with no main effects. For each group, cells were broken down by which condition elicited the strongest firing. C. Correlation between size (big − small/big + small) and delay (short−long/short+long) effects averaged across direction (odor onset to odor port exit). Data was taken after learning (last 10 trials for each condition within each block).

Figure 3. Activity of both increasing- and decreasing-type neurons exhibited an interaction between expected value and direction

A. Each row represents the averaged normalized firing over time during the trial. Cells are sorted based on strength of firing during time in the odor port. How hot the color is depicts the strength of the normalized firing rate. B. Distribution of firing rate indices (poke epoch-baseline/poke epoch + baseline) indicating the difference between activity taken from odor port entry to odor port exit compared to baseline (1 second before nosepoke). Thus, values above and below zero represented increasing- and decreasing-type cells, respectively. Red and blue bars indicate those neurons whose activity was significantly stronger (increasing-type) or weaker (decreasing-type) during the poke epoch compared to baseline (ttest; p < 0.05). C. Results of a two-factor anova with value and direction as factors (p < 0.05) for increasing-type (top panel) and decreasing-type (bottom panel) cells. Firing rate was taken from odor onset to odor port exit. The height of each bar indicates the percentage of neurons that exhibited a main effect of value with no interaction effect, a main effect of direction with no interaction effect or an interaction between the value and direction with no main effects. For each group, cells are broken down by which condition elicited the strongest firing. D. Correlation between size (big − small/big + small) and delay (short−long/short+long) effects averaged across direction (odor onset to odor port exit) for increasing (gray circles) and decreasing-type (black diamonds) cells. Data was taken after learning (last 10 trials for each condition within a block of trials.).

Figure 4. Activity in SNr was modulated by expected reward prior to odor onset

A–B. Curves representing population firing during performance for increasing (n = 147) and decreasing type (n = 44) neurons after learning (last 10 trials for each condition within each block). In this plot, for each neuron, direction and outcome were referenced to the max response before averaging, thus by definition, activity was higher in the preferred outcome/preferred direction (solid black) after odor onset (solid black = preferred outcome/preferred direction; solid gray = non-preferred outcome/preferred direction; dashed black = preferred outcome/non-preferred direction; dashed gray = non-preferred outcome/non-preferred direction). Data are aligned on odor onset; nosepoke occurred 500 ms prior. C–D. Distributions reflecting the difference between pre-cue firing (500 ms; gray bar) when the preferred outcome was in the cell’s preferred direction averaged across response direction. Thus, the x-axis reflects the difference between the average firing rate on the cell’s preferred outcome/preferred direction conditions and cell’s nonpreferred outcome/nonpreferred direction conditions (in both cases the cell’s preferred outcome was in the cell’s preferred direction) minus the average firing rate on preferred outcome/nonpreferred direction conditions and nonpreferred outcome/preferred direction conditions (in both cases the cell’s preferred outcome was opposite the cell’s preferred direction) divided by the sum of the two. Black bars represent the number of neurons that showed a significant difference between these conditions (ttest; p < 0.05).

Figure 5. Activity in SNr was correlated with motor output

A and C. Curves representing activity during fast and slow movements in preferred and non-preferred directions for neurons that showed a negative (A and B; n = 66) and positive correlation (C and D; n = 22) between firing rate (1s prior to the response) and reaction time (odor port exit minus odor offset). See text for details. Plots represent the fastest and slowest 25% of trials for each direction. B and D. Distribution of firing rate indices (poke epoch−baseline/poke epoch + baseline) indicating the difference between activity taken from odor port entry to odor port exit compared to baseline (1s before nosepoke). Thus, values above and below zero represented increasing- and decreasing-type cells, respectively. Black bars indicated those neurons whose activity was significantly stronger (increasing-type) or weaker (decreasing-type) during the poke epoch compared to baseline (ttest; p < 0.05). E. Correlation between directional indices (contra−ipsi/contra+ipsi) for movements cued by free- (x-axis) and forced-choice (y-axis) odors. Activity was taken from odor onset to odor port exit.