Three Rostromedial Tegmental Afferents Drive Triply Dissociable Aspects of Punishment Learning and Aversive Valence Encoding

Abstract

Persistence of reward seeking despite punishment or other negative consequences is a defining feature of mania and addiction, and numerous brain regions have been implicated in such punishment learning, but in disparate ways that are difficult to reconcile. We now show that the ability of an aversive punisher to inhibit reward seeking depends on coordinated activity of three distinct afferents to the rostromedial tegmental nucleus (RMTg) arising from cortex, brainstem, and habenula that drive triply dissociable RMTg responses to aversive cues, outcomes, and prediction errors, respectively. These three pathways drive correspondingly dissociable aspects of punishment learning. The RMTg in turn drives negative, but not positive, valence encoding patterns in the ventral tegmental area (VTA). Hence, punishment learning involves pathways and functions that are highly distinct, yet tightly coordinated.

Keywords: RMTg; VTA; aversion; dopamine; habenula; learning; parabrachial; prefrontal; punishment; valence.

Figure 1.. Rostromedial tegmental (RMTg) neurons respond to affective stimuli.

(A-B) Animals used for RMTg recordings initially underwent Pavlovian training until they achieved >85% accuracy discriminating reward versus neutral cues. (C) During recordings, cued and uncued shock trials were also added, along with occasional reward omissions. (D) Electrode placements were determined with respect to RMTg boundaries delineated by FOXP1 immunostaining. (E-H) Raster plots and histograms of representative RMTg neuron responses to reward, cued shock, reward omission, or uncued shock trials. (I-M) Averaged responses and proportions of RMTg neurons that were inhibited by reward cues and excited by shock predictive cues, shocks (cued or uncued) and reward omission.

Figure 2.. The lateral habenula (LHb) contributes to RMTg activations by aversive prediction errors.

(A-B) We recorded RMTg neuron responses to rewards, shocks, and their predictive cues before and after pharmacological inhibition of the LHb (via baclofen/muscimol cocktail infusion) during the latter half of recording sessions. (C-D) For optogenetic inhibition, virus expressing inhibitory eArch 3.0 was bilaterally injected into the LHb and an optical fiber implanted in the RMTg along with the recording electrode. Laser was turned on during randomly selected trials and remained off in others. (E-I) Time course graphs aligned to cues or outcomes (as denoted above each graph), along with adjacent bar graphs of average firing during intervals indicated by shaded windows or black bars, show that LHb inactivation selectively attenuated RMTg responses to unexpected cues and shocks (grey shaded windows in E, G), and unexpected reward omission (black bar in I), all of which represent “worse than expected” prediction errors (p=0.1069 and p<0.0001 for reward cue and reward omission; p=0.0871 and p=0.001, p=0.8615 and p=0.9794, p=0.074 and p=0.0025 for early and late of shock cue, cued shock, and free shock, respectively). (J-L) Optogenetic inhibition of LHb inputs to the RMTg selectively decreased the same responses. (M) Uncued and cued shocks elicited similar excitations during an “early” time window after stimulus onset (0–30ms, p > 0.05), while uncued shocks produced a much larger response during a “late” time window (30–100ms, p<0.0001), consistent with an aversive prediction error. (N) Some trials used a double cue consisting of two consecutive auditory tones (4kHz and 8kHz) predicting a subsequent shock, making the second cue fully predicted by the first cue. (O) Both cues elicited similar early phase firing, but the first (unpredicted) cue elicited much greater late phase activation (p=0.889 and p=0.005), again consistent with an aversive prediction error. All p-values are from paired t-tests and repeated measures two-way ANOVA, with Holm-Sidak multiple comparisons post-hoc tests. Brown and grey shaded areas: early and late phase response windows. Data for each individual neuron is shown in Figures S2A–E, and individual animal histology in S2F.

Figure 3.. The prelimbic cortex (PL) and parabrachial nucleus (PBN) drive RMTg activations by predictive cues and shocks, respectively.

(A, G, K, Q) Illustrations of pharmacological and optogenetic inhibition. Trial designs were the same as in Figures 2A, C. (B-F, H-J) PL inactivation selectively attenuated RMTg early phase (0–30ms, brown shaded stripe) responses to shock cues and reward cues, but not other stimuli (p=0.0038 and p=0.2777, p=0.1331 and p=0.5463, p=0.4690 and p=0.5075 for early and late components of shock cue, cued shock, and free shock, respectively; p=0.0264 and p=0.3924 for reward cue and reward omission). (L-P, R-T) PBN inactivation selectively attenuated RMTg early phase responses to shocks, without affecting other responses (p=0.1484 and p=0.4233, p=0.0009 and p=0.8003, p<0.0001 and p=0.2680, for early and late responses to shock cue, cued shock, and uncued shock, respectively; p=0.2011 and p=0.8521 for reward cue and reward omission). Neurons that were not inhibited by reward cues or not activated by shock cues, shocks, or reward omission were excluded from analyses in Figures 2 and 3, but data for all neurons, along with histology, are shown in Figures S2, S3, and S4. All p values are from paired t-tests and repeated measures two-way ANOVA, with Holm-Sidak multiple comparisons post-hoc tests. Brown and grey shaded areas: early and late phases.

Figure 4.. PL, PBN, and LHb inputs to the RMTg modulate reward seeking under punishment during distinct phases of decision-making.

(A, B) Behavioral paradigm in which lever pressing for food reward was punished by progressively increasing footshock intensity until a behavioral breakpoint is reached. (C) Schematic of optogenetic procedures in which AAV expressing eArch3.0 is injected into PL, PBN, or LHb, and optical fiber is targeted to RMTg. (D) Photomicrograph of eArch 3.0-expressing terminals (red) intermingled among RMTg neurons identified by FOXP1 immunostaining (green). (E) Laser light (green horizontal bar) was delivered during either the “decision phase” or “shock phase” of the task. Laser delivery during footshock (“synchronized” condition) is compared with control trials in which laser is delivered just before/after footshock (“desynchronized”). (F) Optogenetic inhibition of PL or PBN projections to RMTg increased shock breakpoint when light was delivered during the decision phase or shock phase, respectively, but not vice versa. Inhibition of LHb projection to RMTg was without effect (for decision phase, interaction of pathway × inactivation: p=0.04, with post-hoc tests showing p=0.002 for PL and p=0.923 for PBN and LHb; for shock phase, interaction of pathway × inactivation: p=0.014, with post-hoc tests showing p=0.0019 for PBN, p=0.888 for PL and LHb). (G) Optogenetic inhibition of LHb projection to RMTg increased shock breakpoints during both the decision phase and shock phase when shocks were delivered at 50% probability, even though no effect had been seen earlier with shocks delivered at 100% probability (for decision phase, interaction: p=0.038, post-hoc: p=0.014 and p>0.999 for LHb and no virus groups; for shock phase, interaction: p=0.058, post-hoc: p=0.04 and p=0.7575 for LHb and no virus groups). All status use repeated measures two-way ANOVA, followed by Holm-Sidak multiple comparison post-hoc test.

Figure 5.. Selective ablation of VTA-projecting RMTg neurons abolishes VTA neuron inhibition by aversion-related signals, but not excitation to reward-related signals.

(A) Schematic of viral injections of AAV-Cre into the VTA, and AAV-FLEX-taCasp3 into the RMTg, selectively ablating VTA-projecting RMTg neurons. (B) Quantitation of RMTg FOXP1-positive cells and VTA TH-positive cells shows reduction of FOXP1 ipsilateral to AAV-Cre injection, compared with the contralateral side. Injections of FLEX-taCasp3 into RMTg were placed 1.9mm caudal to AAV-Cre injections, minimizing spread to the rostral RMTg and VTA. Scale bar: 50μm. (C) Schematic of recording sessions in which distinct auditory tones are followed by sucrose at 60%, 90%, or 100% probability, or shock at 100% probability. (D) Heatmaps of individual VTA neuron responses to reward trials. (E, F) Both sham and lesion groups show neurons that are rapidly (type I) or slowly (type II) activated by reward cues. Responses to reward cues or rewards were not affected by ablation at any of the three reward probabilities (For reward cue response, p=0.99 for 60%, 90%, and 100%, sham and lesion group; p=0.99 and p=0.92, and p=0.99 and p=0.77 for 60% vs. 90% and 60% vs. 100% in sham and lesion group, respectively. For reward response, p=0.92 for 60%, 90%, and 100% compared between sham and lesion group; p=0.01 and p=0.001, and p=0.03 and p=0.12 for 60% vs. 90% and 60% vs. 100% in sham and lesion group, respectively, two-way ANOVA, with Holm-Sidak multiple comparisons test). (G, H) Type I VTA neurons were inhibited by footshocks, shock-predictive cues, and 10% reward omission. Ablation of VTA-projecting RMTg neurons eliminated all three of these inhibitions (blue traces/symbols) (shock: p=0.008, cue: p=0.044; two-way ANOVA, with Holm-Sidak multiple comparisons test; 10% omission: p=0.022, paired t-test for firing rates during analysis windows shown with black bars).

Figure 6.. RMTg neurons broadcast punishment signals to multiple targets.

(A) Schematic of experimental design in which we injected retrograde tracer CTb into either the VTA or SNC, and induced cFos with either footshocks or shock-predictive auditory cues. (B) Immunostaining of CTb injection sites in VTA and SNC. Scale bar: 500μm. (C) Representative photomicrographs of CTb and cFos in the RMTg. (D) Shocks and shock cues greatly increased cFos in RMTg versus unstimulated animals, which showed extremely low RMTg cFos levels. (E) Shock- and shock-cue activated RMTg neurons were enriched in both VTA-projecting, and to a lesser extent SNC-projecting, subpopulations.

Figure 7.. Summary of functions subserved by discrete RMTg afferents.

RMTg activations by reward-and punishment-predictive cues, footshocks, and “worse than expected” prediction errors are dependent on triply dissociable inputs from PL (red symbols/lines), PBN (blue symbols/lines), and LHb (orange symbols/lines), respectively, that drive corresponding aspects of punishment learning. Information about all negative motivational stimuli is further transmitted from RMTg to the VTA (green symbols/lines), where it is combined with information about reward-related stimuli. Grey symbols represent neurons that do not project to the same target as the colored symbols, reflecting heterogeneity of projection targets (and likely function) in these regions.