Opponent control of reinforcement by striatal dopamine and serotonin

Abstract

The neuromodulators dopamine (DA) and serotonin (5-hydroxytryptamine; 5HT) powerfully regulate associative learning^1-8. Similarities in the activity and connectivity of these neuromodulatory systems have inspired competing models of how DA and 5HT interact to drive the formation of new associations^9-14. However, these hypotheses have not been tested directly because it has not been possible to interrogate and manipulate multiple neuromodulatory systems in a single subject. Here we establish a mouse model that enables simultaneous genetic access to the brain's DA and 5HT neurons. Anterograde tracing revealed the nucleus accumbens (NAc) to be a putative hotspot for the integration of convergent DA and 5HT signals. Simultaneous recording of DA and 5HT axon activity, together with genetically encoded DA and 5HT sensor recordings, revealed that rewards increase DA signalling and decrease 5HT signalling in the NAc. Optogenetically dampening DA or 5HT reward responses individually produced modest behavioural deficits in an appetitive conditioning task, while blunting both signals together profoundly disrupted learning and reinforcement. Optogenetically reproducing DA and 5HT reward responses together was sufficient to drive the acquisition of new associations and supported reinforcement more potently than either manipulation did alone. Together, these results demonstrate that striatal DA and 5HT signals shape learning by exerting opponent control of reinforcement.

Extended Data Fig. 1:. DAT-Cre^+/−;SERT-Flp^+/− mice enable orthogonal and specific access to VTA^DA and DR^5HT neurons

a, Surgical strategy to validate orthogonality of genetic access to VTA^DA and DR^5HT neurons in DAT-Cre^+/−;SERT-Flp^+/− mice. b, Example image showing negligible Flp-dependent EYFP expression in the VTA. c-d, Example images of the DR showing Cre-dependent mCherry expression is restricted to DR^DA neurons (c) and is not observed in DR^5HT neurons (d). e, Surgical strategy for control experiments to validate the specificity of our viral targeting strategy. f-g, example images showing negligible Cre-dependent mCherry expression in the VTA in the absence of Cre (f) and negligible Flp-dependent EYFP expression in the DR in the absence of Flp (g). In a-d, n = 1 mouse. In e-f, n = 1 mouse.

Extended Data Fig. 2:. Overlap between VTA^DA and DR^5HT axons varies across limbic regions

a, Surgical strategy for VTA^DA and DR^5HT axon tracing experiments. b, Example images showing labeled VTA^DA and DR^5HT axons in sagittal sections (top). Insets (center, bottom) correspond to the boxed regions in the top images. c, Relative density of VTA^DA (left) and DR^5HT (right) axons across limbic regions. d, Background subtracted (left) and segmented (center) images showing VTA^DA and DR^5HT axons in the anterior NAc. Insets (right) show magnified views of the corresponding boxed areas in the left and center images. e, same as d, but for the posterior NAc. f, same as d, but for the anterior BLA. g, same as d, but for the posterior BLA. h, same as d, but for the Ant Ctx. i, Relative colocalization between VTA^DA and DR^5HT axons across the regions shown in d-h. In c and i, n = 5 mice. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Extended Data Fig. 3:. NAc-projecting DR^5HT neurons are distinct from CeA- and OFC- projecting DR^5HT subsystems

a, Surgical strategy for retrograde labeling of projection-defined DR^5HT subsystems. b, Example images of the DR showing retrogradely labeled neurons (posterior DR image reproduced here from Fig. 1m for comparison). c, Percentage of OFC-, CeA-, and NAc- projecting DR neurons that are TpH+. d, Pie graphs showing the fraction of OFC- (left), CeA- (center), and NAc- (right) projecting DR^5HT neurons that send axon collaterals to the other two target regions. e, NAc- projecting DR^5HT neurons are a distinct population from then CeA- and OFC- projecting DR^5HT subsystems. f, Distributions of OFC- (left), CeA- (center), and NAc- (right) projecting DR^5HT neurons across the DR’s anteroposterior axis. g, same as f, but across the dorsomedial (dm), ventromedial (vm), and lateral (l) subregions shown in b. h, Injection strategy (top) and example injection site images (bottom) for retrograde tracing control experiments. I, Example images showing retrogradely labeled cells in the DR. j, When all three retrograde tracers were injected together into the same target structure, the vast majority of labeled cells in the DR were positive for all three tracers. In a-g, n = 3 mice. In h-j, n = 2 mice. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Extended Data Fig. 4:. Inverse VTA^DA and DR^5HT axon reward responses are consistent across mice and are not explained by motion

a, Optical fiber tip placements for mice used in two-color axon photometry experiments. b, RCaMP2 (VTA^DA axon) recordings from individual mice aligned to CS-onset (top) or reward consumption (bottom) early (left) and late (right) in training. c, Same as b, but for GCaMP6 (DR^5HT axon) recordings. d, Average GCaMP6, RCaMP2, and UV photometry traces from an example mouse (n = 25 trials from 1 mouse) aligned to shock onset. Top, demodulated and mean subtracted traces before motion correction, z-scoring, or smoothing; center, Z-scored GCaMP6 traces from the same session with and without motion correction; bottom, Z-scored RCaMP2 traces from the same session with and without motion correction (see Methods for motion correction details). e, same as d, but for photometry traces aligned to reward consumption (n = 35 trials from 1 mouse). Example traces in d-e are from the same mouse shown in Fig. 2b and Fig. 2k–l.

Extended Data Fig. 5:. Fiber placement validation and additional analyses for GRAB sensor recording experiments in NAc^pmSh

a, Optical fiber tip placements for mice used in GRAB-DA (top) and GRAB-5HT (bottom) experiments in the NAc^pmSh. b, GRAB-DA recordings aligned to reward consumption showing the average response across trials for each mouse. c, Same as b, but for GRAB-5HT. d-e, GRAB-5HT recordings aligned to reward consumption during days 1–3 (d) and 5–7 (e) of a task where rewards were delivered randomly and without any predictive cues (Data are shown as mean +/− s.e.m.). For all panels, n = 5 mice per group.

Extended Data Fig. 6:. Optical fiber placement validation and control assays for loss-of-function experiments in the NAc^pmSh

a-d, Example images of the injection sites (DR, top left; VTA, top right) and optical fiber implantation sites (bottom) for the EYFP/EYFP (a), EYFP/NpHR (b), ChR2/EYFP (c), and ChR2/NpHR (d) groups. e-h, Optical fiber tip placements for mice in the EYFP/EYFP (e), EYFP/NpHR (f), ChR2/EYFP (g), and ChR2/NpHR (h) groups. i, Percent change in velocity during the light-on epochs relative to the light-off epochs in the open field test. j, Difference score for time spent on each side of the chamber in the RTPP task. In i-j: EYFP/EYFP, n = 9 mice; NpHR/EYFP, n = 8 mice; EYFP/ChR2, n = 7 mice; NpHR/ChR2, n= 9 mice. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Extended Data Fig. 7:. GRAB sensor validation of the gain- and loss- of function manipulations of VTA^DA and DR^5HT reward responses

a, Viral strategy enabling VTA^DA stimulation, VTA^DA inhibition, and GRAB-DA recordings in the same mouse (top); and, viral strategy enabling DR^5HT stimulation, DR^5HT inhibition, and GRAB-5HT recordings in the same mouse (bottom). b, GRAB-DA recordings aligned to sucrose consumption alone (gray, n = 913 trials from 11 mice) or sucrose consumption with VTA^DA inhibition (red, n = 1380 trials from 11 mice) in the same mice (top); and, GRAB-5HT recordings aligned to sucrose consumption alone (gray, n = 539 trials from 2 mice) or sucrose consumption with DR^5HT stimulation (blue; n = 680 trials from 2 mice) in the same mice (bottom). c, GRAB-DA (top) recordings aligned to sucrose reward consumption (gray, n = 274 trials from 5 mice) or to the onset of VTA^DA stimulation (blue, n = 779 trials from 5 mice) in the same mice (top); and, GRAB-5HT recordings aligned to sucrose reward consumption (gray, n = 1006 trials from 12 mice) or to the onset of DR^5HT inhibition (red, n = 1872 trials from 12 mice) in the same mice (bottom). Data are shown as mean +/− s.e.m.

Extended Data Fig. 8:. Optical fiber placement validation and control assays for gain-of-function experiments in the NAc^pmSh

a-d, Example images of the injection sites (DR, top left; VTA, top right) and optical fiber implantation sites (bottom) for the EYFP/EYFP (a), NpHR/EYFP (b), EYFP/ChR2 (c), and NpHR/ChR2 (d) groups. e-h, Optical fiber tip placements for mice in the EYFP/EYFP (e), NpHR/EYFP (f), EYFP/ChR2 (g), and NpHR/ChR2 (h) groups. Two mice in the NpHR/EYFP group died before their brains could be collected for histology. i, Percent change in velocity during the light-on epochs relative to the light-off epochs in the open field test (n = 6 mice per group). j, Number of trials of each type obtained during days 1–9 of training in the optogenetic conditioning task shown in Fig. 4m–p (n = 10 mice per group). Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Extended Data Fig. 9:. Optical fiber placement validation and individual mouse traces for GRAB sensor recording experiments in NAc^core

a-b, Optical fiber tip placements for mice used in GRAB-DA (a) and GRAB-5HT (b) experiments in the NAc^core. One mouse in the GRAB-5HT group died before its brain could be collected for histology. c, GRAB-DA recordings aligned to reward consumption showing the average response across trials for each mouse (n = 3 mice). d, Same as c, but for GRAB-5HT (n = 4 mice).

Extended Data Fig. 10:. Optical fiber placement validation for loss-of-function experiments in the NAc^core

a-b, Example images of the injection sites (DR, left; VTA, center) and optical fiber implantation sites (right) for the EYFP/EYFP (a), and ChR2/NpHR (b) groups. c-d, Optical fiber tip placements for the EYFP/EYFP (c), and ChR2/NpHR (d) groups.

Fig. 1:. Mapping convergent DA and 5HT inputs to limbic structures involved in learning

a, Schematic describing the generation of DAT-Cre^+/−;SERT-Flp^+/− mice enabling simultaneous and independent genetic access to DA and 5HT neurons. b, Viral strategy for labeling VTA^DA and DR^5HT neurons in a single mouse. c, Example sagittal section depicting mCherry-expressing neurons in the VTA and EYFP-expressing neurons in the DR. d-e, Example images showing colocalization between Cre-dependent mCherry and TH in the VTA. f, Cell-type specificity quantification for VTA^DA neurons in DAT-Cre^+/−;SERT-Flp^+/− mice (n = 3 mice). g-h, Example images showing colocalization between Flp-dependent EYFP and TpH in the DR. i, Cell-type specificity quantification for DR^5HT neurons in DAT-Cre^+/−;SERT-Flp^+/− mice (n = 3 mice). j, Example images of VTA^DA and DR^5HT inputs to limbic structures. k, Relative colocalization between VTA^DA and DR^5HT axons across limbic regions (left) and striatal subregions (right; n = 5 mice). l, Left, injection strategy to label projection-defined DA subsystems. Right, example image showing retrogradely labeled VTA^DA neurons. m, Left, injection strategy to label projection-defined 5HT subsystems. Right, example image showing retrogradely labeled DR^5HT neurons. Note the lack of colocalization between Ctb-488 and the other two tracers in l-m. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Fig. 2:. Convergent DA and 5HT inputs to NAcpmSh show inverse responses to rewards

a, Surgical strategy to record VTA^DA and DR^5HT axon calcium activity in the NAc. b, Confocal image of the recording site from an example mouse. c, Schematics of the fiber photometry system (left) and Pavlovian conditioning task (right). d, By the end of training, mice had acquired the CS-US association as indicated by a decreased latency to collect rewards following CS-onset (left) and an increased number of anticipatory port entries made during the first 5 s of the CS before US delivery (right; n =5 mice). e-j, Population recordings and max/min Z-score quantifications of VTA^DA and DR^5HT axon calcium activity aligned to CS onset (top) or reward consumption (bottom) during early (left) and late (right) training. Neither VTA^DA nor DR^5HT axons showed a CS response at any stage of training (e-g). Late in training, VTA^DA axons were excited by rewards while DR^5HT axons were inhibited (h-j). k-l, Simultaneously recorded VTA^DA and DR^5HT axon calcium responses in an individual mouse during late training aligned to CS-onset (k) and reward consumption (l). m, There was no correlation between the relative timing of the RCaMP2 max and GCaMP6 min (left), but we observed a negative correlation between the magnitude of the RCaMP2 max and the GCAMP6 min (right). n, Surgical strategy (left) and example image of the recording site (right) for GRAB sensor recordings in the NAc^pmSh o-p, DA release increased (o) and 5HT release decreased (p) following reward consumption. In d-j and m, n = 5 mice. In o-p, n = 5 mice per group. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Fig. 3:. Blunting convergent DA and 5HT reward responses disrupts learning and reinforcement

a, Optogenetic strategy to blunt VTA^DA and/or DR^5HT reward responses during learning. b, Images of injection/implantation sites from an example mouse. c, Schematic of the Pavlovian conditioning task. d, Number of rewards obtained over training. Inset, average rewards across days. e, Number of port entries over training. Inset, port entries on the final day of training. f, Median latency to enter the port after CS-onset over training. Inset, median latency across days. g, Probability of occupying the reward port as a function of time within a trial. Inset, average probability during the CS period. h, Baseline-normalized probability of occupying the reward port as a function of time within a trial. Inset, average normalized probability during the CS period. f, Percentage of time that mice occupied the port during each period of a trial, relative to the baseline period. j, Number of port entries during an extinction session. k, Latency to enter reward port after CS-onset during an extinction session. l-m, Same as h, but during an extinction session. n, Percentage of time that mice occupied the port during first 5 trials of the extinction session. o, Distance traveled in the open field. p, Time spent per chamber in the real-time place preference test. q, Average body weight during training days. r, Amount of sucrose solution consumed during a free-access reward task. In d-n and q: EYFP/EYFP, n = 10 mice; NpHR/EYFP, n = 8 mice; EYFP/ChR2, n = 8 mice; NpHR/ChR2, n= 9 mice. In o-p: EYFP/EYFP, n = 9 mice; NpHR/EYFP, n = 8 mice; EYFP/ChR2, n = 7 mice; NpHR/ChR2, n= 9 mice. In r: EYFP/EYFP, n = 9 mice; NpHR/EYFP, n = 8 mice; EYFP/ChR2, n = 8 mice; NpHR/ChR2, n= 9 mice. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Fig. 4:. Integration of opponent DA and 5HT reward responses drives new learning

a, Optogenetic strategy to reproduce VTA^DA and/or DR^5HT reward responses. b, Images of injection/implantation sites from an example mouse. c, Schematic of the CPP tasks. d-e, VTA^DA excitation and DR^5HT inhibition together, but not either manipulation alone, produced CPP. f-h, Neither DR^5HT inhibition alone (f) nor VTA^DA stimulation alone (g) produced CPP in the same mice that previously showed CPP for both manipulations together (h). Purple bars in h represent the same data as the purple bars in e. i, VTA^DA stimulation together with DR^5HT inhibition produced a greater real-time place preference than either manipulation alone. j, VTA^DA stimulation alone, or together with DR^5HT inhibition, increased locomotion in the open field test. k, Schematic of the optogenetic conditioning paradigm with three CS- (compound sound and port light cues) US- (VTA^DA stimulation and/or DR^5HT inhibition) pairs. l, Mice acquired conditioned approach responses to CSs paired with VTA^DA stimulation alone, DR^5HT inhibition alone, or both manipulations together (left), but conditioned responses were more accurate for the CS paired with both manipulations together. m, After training, CSs paired with VTA^DA stimulation alone, DR^5HT inhibition alone, or both manipulations together all functioned as conditioned reinforcers, but responding was above chance level only for the CS paired with both manipulations together. n, During the primary reinforcement test, mice preferred VTA^DA stimulation and DR^5HT inhibition delivered together to either manipulation alone. In d-i, n = 6 mice per group. In k-l: EYFP/EYFP, n = 6 mice; EYFP/NpHR, n = 6 mice; ChR2/EYFP, n = 5 mice; ChR2/NpHR, n = 6 mice. In n-p, n = 10 mice. In j, m-n dashed lines represent chance levels. Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.

Fig. 5:. Opponent control of reinforcement by DA and 5HT generalizes to the NAc^core

a, Surgical strategy to label DR^5HT neurons projecting to different NAc subregions. a, Example image of the injection sites. c, Example images of retrogradely labeled DR^5HT neurons showing colocalization between NAc^pmSh-, NAc^core-, and NAc^latSh- projecting subpopulations (n = 2 mice). d-e, Surgical strategy (d) and example image of the recording site (e) for GRAB sensor recordings in the NAc^core. f-g, DA release increased (f, n = 3 mice) and 5HT release decreased (g, n = 4 mice) following reward consumption. h, Optogenetic strategy to reproduce VTA^DA and/or DR^5HT reward responses in the NAc^core. i, Images of injection/implantation sites from an example mouse. j, Schematics of the RTPP tasks. k, DR^5HT inhibition alone did not produce RTPP (EYFP/EYFP, n = 5 mice; ChR2/NpHR, n = 6 mice). l, VTA^DA stimulation drove RTPP in the ChR2/NpHR group (right, n = 6 mice) but not in EYFP/EYFP controls (left, n = 5 mice). m, Mice in the ChR2/NpHR group (right, n = 5–6 mice), but not the EYFP/EYFP group (left, n = 5 mice), preferred VTA^DA stimulation together with DR^5HT inhibition compared to VTA^DA stimulation alone. n, Difference score for the red-light only RTPP experiment in k. o, Same as n but for the 12 mW experiment in l. p, Same as n, but for the 3 mW experiment in m. In n-p: EYFP/EYFP, n = 5 mice; ChR2/NpHR, n = 6 mice. q-r, In a 4-choice RTPP task, mice showed a within-session place preference for VTA^DA stimulation delivered together with DR^5HT inhibition compared to either manipulation alone (n = 5 mice). Data are shown as mean +/− s.e.m. and significance is denoted as *P<0.05, **P<0.01, ***P<0.001, ****P<0.0001. See Supplementary Table 1 for statistics.