Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice

Abstract

In choosing between different rewards expected after unequal delays, humans and animals often prefer the smaller but more immediate reward, indicating that the subjective value or utility of reward is depreciated according to its delay. Here, we show that neurons in the primate caudate nucleus and ventral striatum modulate their activity according to temporally discounted values of rewards with a similar time course. However, neurons in the caudate nucleus encoded the difference in the temporally discounted values of the two alternative targets more reliably than neurons in the ventral striatum. In contrast, neurons in the ventral striatum largely encoded the sum of the temporally discounted values, and therefore, the overall goodness of available options. These results suggest a more pivotal role for the dorsal striatum in action selection during intertemporal choice.

Figure 1

Intertemporal choice task and the locations of recorded neurons. A. Spatio-temporal sequences of the intertemporal choice and control tasks. B. Probability of choosing the small-reward target (TS) plotted as a function of the delays for the large-reward (TL) and small-reward (TS) targets in an example session. These data were best fit by the hyperbolic discount function with k=0.23 s⁻¹. Solid and dotted lines indicate the predictions from the best-fitting hyperbolic and exponential discount functions, respectively. Error bars, SEM. C. Locations of neurons recorded in the striatum projected onto coronal and sagittal planes. Colors indicate the variables that significantly modulated the activity of each neuron (DV_sum, sum of temporally discounted values; DV_L/DV_R, temporally discounted value for leftward/rightward target; DV_C/DV_UC, temporally discounted value for chosen/unchosen target). When the neuron encoded multiple variables, the variable with the maximum coefficient of partial determination (CPD) is indicated. The outline of the striatum shown in the coronal plane was obtained from an MR image corresponding to the level indicated by the arrow in the sagittal plane. Dotted lines in the coronal plane, border between the caudate nucleus (CD), putamen, and ventral striatum (VS); dotted lines in the sagittal plane, ventral tip of the lateral ventricle (LV) and the T-junctions between CD, putamen, and VS, relative to the LV at the level of anterior commissure.

Figure 2

An example neuron in the caudate nucleus encoding the sum of the temporally discounted values for the two targets and their difference. A. Raster plots for trials first grouped by the magnitudes and delays of rewards from the two targets and sorted by the difference in the temporally discounted values (DV) between them. A pair of numbers to the left indicate the reward delays for the two targets with bold typeface used to indicate the delay for the large reward (e.g., “0:4” corresponds to 0 and 4 s delays for leftward small-reward target and rightward large-reward target). Blue and black rasters indicate the trials in which the animal chose the left and right targets, respectively. Colored rectangles and vertical line segments to the right indicate a set of trials grouped together to calculate average activity shown in B. B. Spike density functions (SDF; top) and firing rates during the cue period (bottom) averaged according to the difference in the temporally discounted values for the two targets during intertemporal choice (left) and control (right) tasks. Empty (filled) circles and dotted (solid) lines denote the activity in trials in which the animal chose the left (right) target. FDV, fictitious temporally discounted value. C. Firing rates during the cue period averaged according to the sum of the temporally discounted values during the two tasks. D. Firing rates during the cue period averaged according to the difference in the temporally discounted values for the chosen and unchosen targets. Lines in B–D are derived from a regression model (model 1) by fixing the values of other regressors at their means. Asterisks indicate that the relationship was statistically significant (*, p<0.05; **, p<0.001). Error bars, SEM.

Figure 3

Two example neurons in the ventral striatum encoding the sum of the temporally discounted values for the two targets (A–C) or the difference in the discounted values for chosen and unchosen targets (D–F). Same format as in Figure 2B–D.

Figure 4

Population summary of activity related to temporally discounted values in the CD and VS. Scatter plots show the standardized regression coefficients (SRC) associated with the temporally discounted values of the left and right targets (A) or chosen and unchosen targets (B). Circles correspond to the neurons for which the effect of the discounted value was significant for at least one of the variables (p<0.05), whereas squares indicate the neurons in which the effect was not significant for either variable. Circles filled in gray and black indicate the neurons in which the effect was significant for both variables at the significance level of 0.1 and 0.05, respectively. Gray area corresponds to the 95% confidence interval for the correlation coefficient obtained from the permutation test.

Figure 5

Activity related to temporally discounted values during intertemporal choice task (abscissa) and fictitious temporally discounted values during control (ordinate) task for the caudate nucleus (A) and ventral striatum (B). The results are shown separately for the sum of the temporally discounted values for left and right targets (left), their difference (middle), and the difference in the temporally discounted value for chosen and unchosen target (right). Empty circles, neurons showing significant interaction between temporally discounted values and task; black disks, neurons showing only the main effect of temporally discounted values; gray disks, neurons without any significant effects of temporally discounted values.

Figure 6

Time course of neural activity related to the animal’s choice and temporally discounted values. A. Fraction of neurons in CD and VS that significantly modulated their activity according to the sum of the temporally discounted values for left and right targets, their difference, the difference in the temporally discounted values for chosen and unchosen targets, and the animal’s choice. B. Population average of the coefficient of partial determination (CPD) for the same variables. Shaded areas, ±SEM.

Figure 7

Effects of spike width and baseline firing rate. A. Example waveform of a neuron recorded in CD. The spike width (distance between the vertical dotted lines) was 0.125 ms for this neuron. B. Relationship between baseline firing rate and spike width. Colors indicate the variable that modulated the activity of each neuron most strongly according to CPD.