Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons

Abstract

Dopamine neurons encode the difference between actual and predicted reward, or reward prediction error (RPE). Although many models have been proposed to account for this computation, it has been difficult to test these models experimentally. Here we established an awake electrophysiological recording system, combined with rabies virus and optogenetic cell-type identification, to characterize the firing patterns of monosynaptic inputs to dopamine neurons while mice performed classical conditioning tasks. We found that each variable required to compute RPE, including actual and predicted reward, was distributed in input neurons in multiple brain areas. Further, many input neurons across brain areas signaled combinations of these variables. These results demonstrate that even simple arithmetic computations such as RPE are not localized in specific brain areas but, rather, distributed across multiple nodes in a brain-wide network. Our systematic method to examine both activity and connectivity revealed unexpected redundancy for a simple computation in the brain.

Figure 1. Rabies-virus-infected putative dopamine neurons show aspects of reward prediction error signals

(A) Probabilistic reward association task. Odor A and B predicted reward with different probabilities (90 and 50%, respectively). Odor C predicted no outcome. Odor D predicted air puff with 80% probability. (B) Experimental design and configuration of electrophysiological recording from rabies virus-infected VTA neurons. (C) Anticipatory licking behavior (1–2 s after odor onset) after rabies injection (mean ± SEM). The dash line separated the data into early and late days in D and E. For all days plotted, the strength of anticipatory licking follows odor A > odor B > odor C (p < 0.001, Wilcoxon signed rank test). (D) Average firing rates of rabies infected putative dopamine neurons during the early (5–9 days, top) and late (10–15 days, middle) periods after rabies virus injection. Putative dopamine neurons that were infected by AAV were shown in the bottom. Only rewarded trials are shown. (E) Quantifications of RPE signals (rabies group at 10–15 days after rabies injection versus AAV control). ***, p < 0.001 (Wilcoxon signed rank test against 0 spikes/s). ** p < 0.01; n.s., p >0.05 (Wilcoxon rank-sum test. Rabies versus AAV control). The edges of the boxes are the 25th and 75th percentiles, and the whiskers extend to the most extreme data points not considered outliers. Points are drawn as outliers if they are larger than Q3+1.5*(Q3-Q1). See also Figure S1.

Figure 2. Each input area contains input neurons with diverse response profiles

(A) Configuration of input neuron recordings. (B) Firing rates of randomly sampled individual identified input neurons from three input areas. (C) Temporal profiles of all input neurons and VTA neurons. Colors indicate an increase (yellow) or decrease (blue) from baseline, as quantified using a sliding–window auROC analysis (time bin: 100 ms, against baseline). Neurons were clustered into five clusters, separated by horizontal red lines. Within each cluster, neurons are ordered by their average responses during 2 s after odor onset. (D) Average firing rates of neurons from each cluster in (C). n, number of neurons. (E) The percentage of neurons that belong to each of the five clusters in individual input areas and VTA. VTA putative GABA and VTA other neurons are cluster 2 and 3 neurons respectively from the AAV control group (see Figure S1) (F) The percentage of variance explained by the first n principle components (PC) in identified inputs and dopamine neurons. PCs were computed for each individual input area and plotted as light green traces. The solid green trace is mean ± SD of traces from all input areas. Subthalamic nucleus has only 7 identified inputs and is excluded from the analysis. See also Figure S2 and S3.

Figure 3. Correlation analysis of dopamine neurons and their inputs

(A) Correlation matrix of firing patterns of dopamine neurons and their input neurons. The similarity of response profiles between pairs of neurons was quantified based on the Pearson’s correlation coefficient. The red lines separate neurons of each brain area. The purple ticks mark the boundaries of different clusters within each area (see Figure 2). Within each area, neurons are ordered in the same way as in Figure 2C. (B) Example firing profiles of dopamine neurons and PPTg neurons (n = 7 each) in 90% reward trials. Note that dopamine neurons’ firing profiles are more similar to one another than PPTg neurons. (C) Histogram of pairwise correlations for all pairs of neurons in each input area and dopamine neurons. The red line indicates the mean pairwise correlation for each area. The grayscale indicates the probability density.

Figure 4. Mixed coding of reward and expectation signals in input neurons

(A–D) Firing rates (mean ± SEM) of example neurons that were categorized as different response types. Neurons are from dorsal striatum (A), lateral hypothalamus (B), RMTg (C) and ventral striatum (D). (E) Flatmap summary (Swanson, 2000) of the percentage of input neurons and control VTA neurons that were classified into each response type. See Experimental Procedures and Figure S4 for criteria. VTA putative GABA and VTA other neurons are cluster 2 and 3 neurons respectively from the AAV control group (see Figure S1). The thickness of each line indicates the number of input neurons in each area (per 10,000 total inputs in the entire brain) as defined at the top right corner (Watabe-Uchida et al., 2012). DS, dorsal striatum; VS, ventral striatum; VP, ventral pallidum; LH, lateral hypothalamus; STh, subthalamic nucleus; DA, dopamine neurons. See also Figure S4.

Figure 5. Input neurons encode reward prediction error signals

(A–D) Firing rates (mean ± SEM) of example neurons that were categorized as encoding different parts of RPE signals. Neurons are from RMTg (A, B), PPTg (C) and ventral pallidum (D). (E) The percentage of input neurons and control VTA neurons that showed RPE signals. See Figure S6 for criteria. The conventions are the same as Figure 4E. (F) The percentage of neurons in each response type for identified input neurons and other (unidentified) neurons. Pure reward type includes neurons with cue responses. Mixed type of responses as well as RPE types are more concentrated in input neurons. * p < 0.05; ** p < 0.01; *** p < 0.001; no labels, p > 0.05 (Chi-squared test, p-values are after Bonferroni correction for multiple comparisons) See also Figure S5 and S6.

Figure 6. Positive and negative responses in inputs

(A) Firing rates (mean ± SEM) of example input neurons (top) and the percentage of neurons (bottom) encoding reward positively and negatively during CS (0–500 ms after odor onset) in identified inputs. Filled bars indicate the percentage of neurons, that monotonically encode reward probability faster than median latency of dopamine neurons (see Table S2, Experimental Procedures). (B) The percentage of neurons encoding reward value positively or negatively during late delay (1500–2000 ms after odor onset) in identified inputs. (C) Firing rates (mean ± SEM) of example neurons (top) and the percentage of neurons (bottom) encoding air puff positively and negatively during CS (0–500 ms after odor onset) in identified inputs. Filled bars indicate the percentage of neurons that distinguish air puff CS from nothing CS faster than median latency of dopamine neurons (see Table S2, Experimental Procedures). (D) The percentage of neurons encoding air puff positively or negatively during late delay (1500–2000 ms after odor onset) in identified inputs. See also Figure S7.

Figure 7. A linear model can reliably extract RPE signals from inputs

(A) In a regularized linear model, dopamine neurons’ responses in the task are modeled as the weighted sum of all input neurons. See Experimental Procedures for details of the model. (B) The linear model fitted dopamine activity (fitted response) and the actual dopamine activity (dopamine response). Inset, distribution of fitted weights. (C) Comparison of quality of fitting using randomly sampled neurons from identified input neurons and other (unidentified) neurons in model fitting (mean ± SD). (D) Comparison of quality of fitting when shuffling the fitted weights using identified input neurons and other neurons (mean ± SD, 500 simulations). To shuffle the data, the weights of two randomly chosen neurons are swapped. “#Pairs swapped” indicates how many times this procedure is repeated for each simulation. (E) Mean ± SEM (100 simulations) of the fitted activity when the fitted weights of input neurons are completely shuffled.