Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors

Abstract

Correlative studies have strongly linked phasic changes in dopamine activity with reward prediction error signaling. But causal evidence that these brief changes in firing actually serve as error signals to drive associative learning is more tenuous. Although there is direct evidence that brief increases can substitute for positive prediction errors, there is no comparable evidence that similarly brief pauses can substitute for negative prediction errors. In the absence of such evidence, the effect of increases in firing could reflect novelty or salience, variables also correlated with dopamine activity. Here we provide evidence in support of the proposed linkage, showing in a modified Pavlovian over-expectation task that brief pauses in the firing of dopamine neurons in rat ventral tegmental area at the time of reward are sufficient to mimic the effects of endogenous negative prediction errors. These results support the proposal that brief changes in the firing of dopamine neurons serve as full-fledged bidirectional prediction error signals.

Figure 1. Task design, fiber placements, and immunohistochemical and electrophysiological verification of Cre-dependent NpHR and eYFP expression in TH (+) neurons in the VTA

a) Top: Illustration of the behavioral task. Bottom: temporal configuration of light inhibition in relative to averaged duration of pellet consumption during reward. b) Fiber implants were localized in the vicinity of eYFP and NpHR expression in VTA. The light shading represents the maximal spread of expression at each level, whereas the dark shading represents the minimal spread. c) Images (left) show that majority of NpHR-expressing neurons (green) also expressed tyrosine hydroxylase (red). Scale: 1 mm. Note that because the image was taken under large field scanning, the signal intensity during acquisition was adjusted to capture the overall brightness of the entire field without ignoring relatively weak yet positive signals. This will inevitably render some area seems to be overpowered by the signal and tip the balance of color detection in the merged image, particularly in the low magnitude image. (Middle) Representative traces show that NpHR-expressing neurons were responsive to light inhibition (shown as green bar). These neurons also expressed TH, confirmed by intracellular labeling and post-hoc TH staining (middle bottom). Spontaneous and evoked firing of NpHR-expressing neurons were interrupted by brief pulses of light inhibition (n = 10 from 3 subjects, firing activity of individual neurons is summarized at the right).

Figure 2. Optogenetic inhibition of TH+ neurons in VTA mimics learning induced by reward over-expectation

Conditioned responding of a) eYFP and b) NpHR rats to the critical auditory cues during the initial conditioning or reminder training (left column), during compound training (middle column), and the subsequent probe tests (right column). Conditioned responding is represented as the percent of time the rats spent in the food cup during the cues. Percent of time in the food cup during the food consumption period during compound training, when TH+ neurons were inhibited, is shown in the bar graph insets. Data from the reward run (1^st and 3^rd rows, Reward), when TH+ neurons were inhibited at the time of reward, is shown in the dark symbols; data from the ITI run (2^nd and 4^th rows, ITI), when TH+ neurons were inhibited in the ITI is shown in the light symbols. N = 8 each group. Vertical bars show S.E.M. n.s., non-significant at > 0.10; *, p < 0.0001 based on stats given in main text. All values in each line plot represent % time rats spent in food cup during the CS after correction for rearing.

Figure 3. Differences in conditioned responding caused by optogenetic inhibition of TH+ neurons in VTA

Differences in conditioned responding of a) eYFP and b) NpHR rats to the critical auditory cues in probe tests after inhibition during reward (black histograms) or after inhibition during the ITI (gray histograms). Scatter plots show data points from the individual rats in each group across the two tests. Difference scores were calculated for each rat as the difference in responding to A2-A1 across all trials in the relevant probe test (using data shown in Fig. 2). Distributions are centered on the difference on the final day of conditioning prior to compound training. The numbers in each panel indicate results of Wilcoxon signed-rank test (p) and the average scores (u). Numbers comparing panels indicate the results of a Wilcoxon rank sum test (p).