Coding of task reward value in the dorsal raphe nucleus

Abstract

The dorsal raphe nucleus and its serotonin-releasing neurons are thought to regulate motivation and reward-seeking. These neurons are known to be active during motivated behavior, but the underlying principles that govern their activity are unknown. Here we show that a group of dorsal raphe neurons encode behavioral tasks in a systematic manner, tracking progress toward upcoming rewards. We analyzed dorsal raphe neuron activity recorded while animals performed two reward-oriented saccade tasks. There was a strong correlation between the tonic activity level of a neuron during behavioral tasks and its encoding of reward-related cues and outcomes. Neurons that were tonically excited during the task predominantly carried positive reward signals. Neurons that were tonically inhibited during the task predominantly carried negative reward signals. Neurons that did not change their tonic activity levels during the task had weak reward signals with no tendency for a positive or negative direction. This form of correlated task and reward coding accounted for the majority of systematic variation in dorsal raphe response patterns in our tasks. A smaller component of neural activity reflected detection of reward delivery. Our data suggest that the dorsal raphe nucleus encodes participation in a behavioral task in terms of its future motivational outcomes.

Figure 1.

Behavioral tasks. A, Memory-guided saccade task with biased reward schedule. B, Visually guided saccade task (VGS) with biased reward schedule. In each task, the animal was required to hold its gaze on a fixation point that appeared at the center of the screen. In the memory-guided saccade task (MGS), a visual target was briefly flashed on the left or right side of the screen. The animal was required to hold fixation until the fixation point turned off and then saccade to the remembered location of the target. In the visually guided task, a visual target appeared that the animal was required to saccade to immediately. Both tasks had a reward schedule with two blocks of trials. In one block of 20–32 trials, the left target was rewarded and the right target was unrewarded; in the next block of trials, their reward values were reversed.

Figure 2.

Activity of three dorsal raphe neurons during the memory-guided saccade task. Neural activity is aligned on fixation point onset (left), target onset (middle), and outcome onset (right). Curves indicate average firing rate on all trials (black), rewarded trials (red), or unrewarded trials (blue). Spiking activity was smoothed with a Gaussian kernel (σ = 20 ms). Black asterisks (**) indicate significantly different activity during the 500–900 ms after fixation point onset compared with a prefixation period 0–400 ms before fixation point onset (p ≤ 0.005, Wilcoxon rank-sum test). Red and blue asterisks (**) indicate significantly different activity for the two reward conditions during a 150–450 ms window after target onset, go onset, or outcome onset (from left to right). A, Neuron that increased its tonic activity during the task and emitted positive reward signals in response to the targets and outcomes. B, Neuron that decreased its tonic activity during the task and emitted negative reward signals in response to the target and outcomes. C, Neuron that did not change its tonic activity during the task and had little or no reward signals.

Figure 3.

Population average activity of dorsal raphe neurons separated by their reward signals in response to the outcome. A–C, Normalized activity is shown for the memory-guided saccade task (MGS, left) and visually guided saccade task (VGS, right), separately for positive-reward cells (A, top), negative-reward cells (B, middle), and non-outcome responsive cells (C, bottom). Neurons were sorted into these categories based on significant reward discrimination during a 150–450 ms window after outcome onset (gray bar on x-axis; p < 0.05, Wilcoxon rank-sum test). The histograms below (C) show the reward discrimination for each neuron, with colors indicating positive-reward cells (red) and negative-reward cells (blue). For the plots of normalized activity, the activity of each neuron was smoothed with a 151 ms running average and normalized to lie between 0 and 1 by computing its ROC area versus the baseline activity of the neuron during the intertrial interval (see Materials and Methods). Thick lines indicate mean normalized activity and the light shaded areas are ±1 SEM. In both tasks, neurons with positive reward discrimination between outcomes had elevated activity during the tasks and positive responses to the rewarded target (A). Neurons with negative reward discrimination between outcomes had suppressed activity during the tasks and negative responses to the rewarded target (B).

Figure 4.

Correlation between dorsal raphe neuron task coding and reward coding. A, Plot of fixation period response (x-axis) versus reward-related response (y-axis) separately for the memory-guided saccade task (MGS, top) and visually guided saccade task (VGS, bottom). The fixation period response was measured as the ROC area for each neuron for discriminating between its firing rate 500–900 ms after fixation point onset versus a prefixation period 0–400 ms before fixation point onset. Reward discrimination was measured during several time windows during the trial (columns): after target onset, after fixation offset (go period), and after outcome onset. Text indicates rank correlation (rho), and asterisks indicate its p value (*p < 0.05; **p < 0.01; ***p < 0.001, permutation test). Dark dots indicate cells with a significant excitation or inhibition during the fixation period (p < 0.05, Wilcoxon rank-sum test). Colored dots indicate cells with significantly higher activity during rewarded trials (red) or during unrewarded trials (blue) (p < 0.05, Wilcoxon rank-sum test). Black lines indicate the line of best fit calculated with type 2 least-squares regression. Neural activity during the fixation period was positively correlated with reward coding during the target, go, and outcome periods. B, Same as A but using absolute ROC area, which ranges between 0.5 (no discrimination) and 1.0 (perfect discrimination), independent of the direction of activity changes or reward discrimination. Neurons with strong responses during the fixation period had significantly stronger reward signals.

Figure 5.

Principal components of dorsal raphe neuron activity. A, B, The first (A) and second (B) principal components of dorsal raphe neural activity profiles during the memory-guided saccade task (MGS, top) and visually guided saccade task (VGS, bottom). Curves represent the normalized firing rate of the principal component during the fixation period (black), after the onset of the rewarded target (red), and after the onset of the unrewarded target (blue), separately for the contralateral-rewarded block (dark colors) and ipsilateral-rewarded block (light colors). The first principal component (A) indicated tonically increased activity during the task and positive-reward coding during the target, memory, and outcome periods. The second component (B) indicated tonically increased activity in response to reward delivery. C, The mean neural activity profile during the memory-guided task (top) and visually guided task (bottom) consisted of phasic responses to the fixation point and targets with no conspicuous tonic activity. D, Percentage of variance in the neural activity profiles explained by the first eight principal components, separately for the true data (black lines) and shuffled datasets (gray lines). Gray error bars indicate the range of percentage variance explained observed in 200 separate shuffled datasets. Only the first two components explained more variance than expected by chance. E, Weight assigned to the first two principal components by each neuron during the memory-guided task (top) and visually guided task (bottom). Each dot represents a single neuron. Because principal components are only specified up to an arbitrary scaling factor, we chose to scale each principal component so that its distribution of single-neuron weights had unit variance. There was no systematic relationship between the weights assigned to the first and second components.

Figure 6.

Little relationship between task-related activity electrophysiological properties. A, The relationship between spike duration (y-axis) and task-related signals (x-axis) including fixation period activity (left column) and the single-neuron weights assigned to the first two principal components (middle and right columns). Data are shown separately for the memory-guided saccade task (MGS, left subcolumns) and visually guided saccade task (VGS, right subcolumns). Each dot is one neuron. Spike waveforms were recorded in a subset of neurons (memory-guided task, n = 43 of 84; visually guided task, n = 70 of 165). Text indicates the rank correlation (rho) and its p value (permutation test, 2000 permutations). The black lines were fit by least-squares linear regression. B, C, Same as A, for the irregularity index (B) and baseline firing rate (C). Overall, most correlations were small in size and did not reach significance (p > 0.05). The exception was a modest tendency for negative correlation between baseline firing rate and the weights of the principal components (C; rho between −0.23 and −0.02), which may be a result of a floor effect on neural activity.