Dopaminergic responses to identity prediction errors depend differently on the orbitofrontal cortex and hippocampus

Abstract

Adaptive behavior depends on the ability to predict specific events, particularly those related to rewards. Armed with such associative information, we can infer the current value of predicted rewards based on changing circumstances and desires. To support this ability, neural systems must represent both the value and identity of predicted rewards, and these representations must be updated when they change. Here we tested whether prediction error signaling of dopamine neurons depends on two areas known to represent the specifics of rewarding events, the HC and OFC. We monitored the spiking activity of dopamine neurons in rat VTA during changes in the number or flavor of expected rewards designed to induce errors in the prediction of reward value or reward identity, respectively. In control animals, dopamine neurons registered both error types, transiently increasing firing to additional drops of reward or changes in reward flavor. These canonical firing signatures of value and identity prediction errors were significantly disrupted in rats with ipsilateral neurotoxic lesions of either HC or OFC. Specifically, HC lesions caused a failure to register either type of prediction error, whereas OFC lesions caused persistent signaling of identity prediction errors and much more subtle effects on signaling of value errors. These results demonstrate that HC and OFC contribute distinct types of information to the computation of prediction errors signaled by dopaminergic neurons.

Figure 1:. Surgical coordinates and extent of ipsilateral hippocampal and orbitofrontal lesions

(a and b) Table gives volumes and coordinates (AP and ML relative to bregma and DV relative to brain surface) of injections. Brain sections illustrate the extent of the maximum (gray) and minimum (black) lesion at each level in HCx (a) and OFCx (b) in the lesioned rats.

Figure 2:. Task design and behavior

(a) Picture of apparatus used in the task, showing the odor port (~2.5 cm diameter) and two fluid wells. (b) Deflections indicate the time course of stimuli (odors and rewards) presented to the rat on each trial. Dashed lines show when a reward was omitted, and solid lines show when reward was delivered. At the start of each recording session, one well was randomly designated to deliver the big reward, which consisted of three drops of flavored milk (chocolate or vanilla). One drop of the other flavored milk was delivered in the other well (block 1). In the second and fourth blocks, the number off drops delivered in the two wells was switched without changing the flavors (value shift). In the third and fifth blocks, the flavors delivered in the two wells were switched without changing the number of drops (identity shift). (c) In all three groups, chocolate- and vanilla-flavored milk was equally preferred in two-flavor choice test (F’s <0.28, p’s > 0.62) and there were no significant main effects or interactions with group (F’s < 0.19, p’s > 0.78). (d – f) Effect of value and identity switches on choice behavior in Control (d), HCx (e) and OFCx (f) groups. Across identity shifts there were no effects of block and no interactions with either group or identity (F’s < 2.02, p’s > 0.16), whereas across value shifts, there were significant main effects of block overall and within each group (F’s > 344.0, p’s < 0.000) and no interactions with either identity or group (F’s < 3.1, p’s > 0.08). (g-l) Reaction time and % correct in response to high and low valued reward of each identity on forced choice trials in control (g), HCx (i) and OFCx (k) groups. All groups showed faster reaction times and higher accuracy when the high valued reward was at stake (F’s > 31.0, p’s < 0.0000), and there were no interactions with either identity or group (F’s < 1.6, p’s > 0.19).

Figure 3:. Identification and waveform features of putative dopamine neurons

(a) Result of cluster analysis based on the half-time of the spike duration and the ratio comparing the amplitude of the first positive and negative waveform segments ((n − p) / (n + p)). Data in left show VTA neurons (n = 408) from the control group, plotted as putative dopamine neurons (gray circles, n = 80), and neurons that classified with other cluster (open circles, n = 328). Data in left show VTA neurons (n = 513) from the HCx group, plotted as putative dopamine neurons (gray circles, n = 110), and neurons that classified with other cluster (open circles, n = 403). Data in right show VTA neurons (n = 412) from the OFCx group, plotted as putative dopamine neurons (gray circles, n = 90), and neurons that classified with other cluster (open circles, n = 322). (b) bar-graphs indicate average amplitude ratio (left) and half duration (right) of putative dopamine neurons in control (open), HCx (gray) and OFCx (black) groups. ANOVA revealed no significant main effect of group in amplitude ratio (F_2,277 = 1.44, p > 0.24) and half duration (F_2,277 = 2.59, p > 0.07).

Figure 4:. Changes in reward-evoked activity of reward-responsive dopamine neurons to changes in reward value.

(a, c and e) Population responses show the difference in firing between early and late trials in response to unexpected reward delivery (blue) and unexpected omission (red) in control (a), HCx (c), and OFCx (e) groups. Gray shadings indicate the time when the reward was manipulated, (b, d and f) Average firing in response to reward delivery (blue) and omission (red) in the first 5 and last 5 trials in control (b), HCx (d) and OFCx (f) groups. Error bars, S.E.M. ANOVA (Reward x Early/Late x Trial) revealed significant main effects of Reward in all groups (Control, F_1,87 = 7.47, p<0.01; HCx, F_1,117 = 8.28, p<0.01; OFCx, F_1,89 = 21.2, p<0.01), and significant interactions between Reward x Early/Late in Control (F_1,87 = 13.1, p<0.01) and OFCx groups (F_1,89 = 12.5, p<0.01). Planned contrasts revealed a significant main effect of Early/Late for reward delivery in Controls (F_1,87 = 9.78, p<0.01) and OFCx (F_1,89 = 10.6, p<0.01), but not in HCx (F_1,117 = 1.18, p>0.05) and a main effect of Early/Late for reward omission in control (Control, F_1,87 =4.47, p<0.05), but not in HCx (F_1,117 = 0.08, p=0.78) or OFCx (F_1,89 = 1.28, p=0.26). (g – i) Distributions of difference scores comparing firing to unexpected reward delivery (left) and omission (right) in the early and late trials in control (g), HCx (h) and OFCx (i) groups. The numbers in each panela indicate results of Wilcoxon singed-rank test (p) and the average difference score (u).

Figure 5:. Changes in reward-evoked activity of reward-responsive dopamine neurons to changes in reward identity

(a, c and e) Difference in population activity between last 5 trials before and first 5 trials after a shift of reward identity for average of 1^st , 2^nd big reward and small reward (purple) and a comparison period where there was no shift taken 0.5 and 1.0s after small reward (green) in control (a), HCx (c) and OFCx (e) groups. Gray shadings indicate the time when the reward was manipulated. (b, d and f) Changes in average firing before and after reward identity shift in control (b), HCx (d) and OFCx (f) groups. Purple lines indicate average firing at the time of the 1^st and 2^nd drops of the big reward and the small reward. Green lines indicate average firing in comparison period after small reward. Error bars, SEM. ANOVA (Shift/No shift x Prev/Early/Late x Trial) revealed a significant main effect of Shift/No shift in all groups (Control, F_1,45 = 15.3, p < 0.01; HCx, F_1,64 = 16.1, p < 0.01; OFCx, F_1,42 = 18.9, p < 0.01), and a significant interaction between Shift/No shift x Prev/Early/Late in Control (F_2,90 = 4.74, p < 0.05), but not in HCx and OFCx (F’s < 0.76, p’s > 0.47). Planned contrasts revealed a significant main effect of Prev/Early/Late in firing after shifts in Control (F_2,90 = 6.40, p < 0.01), but not in HCx (F_2,128 = 0.71 p > 0.49) and OFCx (F_2,84 = 0.08, p > 0.92). There were no significant main effects or interactions in firing in the comparison period in any group (F’s < 72, p’s >0.49). (g – i) Distributions of difference scores comparing firing to first (left), second (middle) drops of big reward and small reward (right) in the last 5 versus first 5 trials before and after an identity shift in control (g), HCx (h) and OFCx (i) groups. The numbers in each panela indicate results of Wilcoxon singed-rank test (p) and the average difference score (u).