Neurons in the ventral striatum exhibit cell-type-specific representations of outcome during learning

Abstract

The ventromedial striatum (VMS) is a node in circuits underpinning both affect and reinforcement learning. The cellular bases of these functions and especially their potential linkages have been unclear. VMS cholinergic interneurons, however, have been singled out as being related both to affect and to reinforcement-based conditioning, raising the possibility that unique aspects of their signaling could account for these functions. Here we show that VMS tonically active neurons (TANs), including putative cholinergic interneurons, generate unique bidirectional outcome responses during reward-based learning, reporting both positive (reward) and negative (reward omission) outcomes when behavioral change is prompted by switches in reinforcement contingencies. VMS output neurons (SPNs), by contrast, are nearly insensitive to switches in reinforcement contingencies, gradually losing outcome signaling while maintaining responses at trial initiation and goal approach. Thus, TANs and SPNs in the VMS provide distinct signals optimized for different aspects of the learning process.

Figure 1. Behavioral Task and Classification of Recorded Neuronal Populations

(A) The T-maze task. (B) Average learning curve for all animals (n = 8) across the four successive training phases: early (AA1) and late (AA2) auditory task acquisition; auditory task over-training (AO); and tactile task acquisition (TA). Error bars represent SEM. (C) Tetrode recording sites. Colors represent sites for individual rats. Anterior-posterior coordinates indicated below. (D) Percentage of recorded VMS units classified as spiny projection neurons (SPNs, blue), high-firing neurons (HFNs, red) and tonically active neurons (TANs, green). (E) Firing rates and median ISIs for recorded units, color-coded by putative cell type, distinguishing three clusters occupying distinct domains. Axes truncated to show regions of overlap. Insert: Axes in log scale to show results for all recorded units. (F) Firing rates and median ISIs of optogenetically identified cholinergic interneurons (n = 12, green) and unclassified units (gray) in ChAT-Cre transgenic mice (n = 6), plotted as in E. (G) Photomicrograph illustrating striatum of ChAT-Cre mouse stained with anti-GFP antibody showing cell bodies (dark brown) and processes (lighter brown) of transfected cholinergic interneurons. (H) Spike raster plot of a cholinergic interneuron silenced by a 500-ms yellow light pulse (yellow shading). (I and J) Spike suppression in 12 individual cholinergic interneurons (I) and average suppression (J) that occurred rapidly in response to the light pulse. Error bars represent SEM. See also Figure S1.

Figure 2. Ensemble Activity of SPNs, TANs and HFNs during Training

(A–C) Pseudocolor plots showing average z-score normalized firing rates on rewarded trials for task-responsive SPNs (A), all recorded TANs (B) and all recorded HFNs (C), plotted in 20-ms bins, relative to baseline firing rates. Successive ±350-ms peri-event windows (BL: baseline, WC: warning click; GT: gate opening; PC: pre-cue; CU: cue onset; TOn: turn onset; TOff: turn offset; G: goal-reaching; PG: post-goal) were abutted in order of occurrence within a trial; continuous time is not shown. Training stages and phases (indicated to left) as defined in text. Number of units compiled for each stage indicated at right. See also Figure S2.

Figure 3. Three Types of SPN Response Recorded during Training

(A–C)Raster plots for click-responsive (A), ramping (B), and reward-responsive (C) SPNs. The average waveform for each unit is shown at right. (D–F) Average z-score normalized activity of click-responsive SPNs (D), ramping SPNs (E), and reward-responsive SPNs (F) across training stages, plotted as in Figure 2. (G–I) Firing rates of click-responsive SPNs (G), ramping SPNs (H), and reward-responsive SPNs (I) in rewarded (blue) and unrewarded (red) trials, averaged across all training stages. Shading indicates SEM. (J) Firing rates of reward-responsive SPNs in rewarded trials, averaged separately for each training phase (color-coded). Dashed line represents the period of licking behavior. (K)Average firing rate of reward-responsive SPNs during a 700-ms period after goal-reaching across training phases. *p < 0.05; **p <0.01. Error bars represent SEM. (L)The percentage of reward-responsive SPNs across training phases. See also Figures S3 and S4.

Figure 4. Reward-Responsive High-Firing Neurons

(A and B) Spike activity of a reward-suppressed HFN (A) and a reward-activated HFN (B). Their average waveforms are shown at right. (C and D) Firing rates of reward-suppressed (C) and reward-activated (D) HFNs in rewarded (blue) and unrewarded (red) trials, averaged across all training stages. Shading indicates SEM. (E and F) Average firing rates of reward-suppressed (E) and reward-activated (F) HFNs in rewarded trials, plotted as in Figure 3J. Data from training phases with fewer than 5 units are not shown. (G and H) The percentage of reward-suppressed (G) and reward-activated (H) HFNs across training phases. *p < 0.05; **p <0.01. See also Figure S4.

Figure 5. Reward-Responsive Tonically Active Neurons

(A–C) Spike activity of three individual reward-responsive TANs recorded in training phases AA2 (A), AO (B) and TA (C) around goal-reaching in either rewarded (top) or unrewarded (bottom) trials. An example from the initial acquisition phase AA1 is not shown because only three TANs were recorded in this phase. The average waveform of each unit is shown at right. (D–F) Average firing rate of all reward-responsive TANs in training phases AA2 (D), AO (E) and TA (F) in rewarded (blue) and unrewarded (red) trials. Shading indicates SEM. (G) Average firing rate of reward-responsive TANs in rewarded trials across training phases, plotted as in Figure 3J. (H) The average firing rate of the reward-responsive TANs on rewarded trials measured in a 300-ms period after goal-reaching across training phase. Error bars show SEM. **p <0.01. (I) Percentage of reward-responsive TANs across training phases. (J) Average firing rate of reward-responsive TANs in response to delayed reward (pump onset). See also Figure S4.

Figure 6. Changes in Activity of Different Neuronal Subtypes during Learning

Schematic drawing illustrating changes in reward-related activity of SPNs (blue), HFNs (red) and TANs (green) across training phases.