Prolonged dopamine signalling in striatum signals proximity and value of distant rewards

Abstract

Predictions about future rewarding events have a powerful influence on behaviour. The phasic spike activity of dopamine-containing neurons, and corresponding dopamine transients in the striatum, are thought to underlie these predictions, encoding positive and negative reward prediction errors. However, many behaviours are directed towards distant goals, for which transient signals may fail to provide sustained drive. Here we report an extended mode of reward-predictive dopamine signalling in the striatum that emerged as rats moved towards distant goals. These dopamine signals, which were detected with fast-scan cyclic voltammetry (FSCV), gradually increased or--in rare instances--decreased as the animals navigated mazes to reach remote rewards, rather than having phasic or steady tonic profiles. These dopamine increases (ramps) scaled flexibly with both the distance and size of the rewards. During learning, these dopamine signals showed spatial preferences for goals in different locations and readily changed in magnitude to reflect changing values of the distant rewards. Such prolonged dopamine signalling could provide sustained motivational drive, a control mechanism that may be important for normal behaviour and that can be impaired in a range of neurologic and neuropsychiatric disorders.

Figure 1. Ramping striatal dopamine signals occur during maze runs

a, b, Baseline subtracted current (a) and dopamine concentration ([DA], b) measured by FSCV in VMS during a single T-maze trial. c, d Trial-by-trial changes in dopamine concentration (c) and velocity (d) relative to goal-reaching. e, f, Dopamine concentration (mean ± s.e.m.) for VMS (e, n = 300 session-averaged recordings from 18 probes across 214 sessions and for DLS (f, n = 262, 13 probes) for correct (blue) and incorrect (red) trials, averaged over all 40-trial sessions.

Figure 2. Ramping dopamine signals proximity to distant rewards

a, Distribution of trial times (from warning click to goal-reaching, n = 3933 trials). b, c, Dopamine release modelled as a function of time elapsed since maze-running onset (b) and as a function of spatial proximity to visited goal (c) for short (purple) and long (orange) trials (see Methods). Vertical lines indicate trial start (red) and end (purple and orange) times. d, Peak dopamine concentration vs. trial time for all ramping trials (n = 2273, Pearson’s R = 0.0004, P = 0.98). e, Experimentally recorded dopamine release (mean ± s.e.m.) in short (n = 327, purple) and long (n = 423, orange) trials. Dopamine peaks at equivalent levels, as in proximity model in c. f, Normalized peak dopamine levels (mean ± s.e.m.) predicted by time-elapsed (red) and proximity (light blue) models, and measured experimental data (dark blue).

Figure 3. Dopamine ramping is sensitive to reward magnitude

a, b, Average dopamine signals from a VMS probe, for consecutive T-maze (a) and M-maze (b) sessions with asymmetric rewards. Asterisks indicate the goal with larger reward. Red arrows (and Switch) indicate reversal of reward amounts. c, Dopamine signals from a different rat running in the S-maze. White arrows indicate run direction. d, Average (± s.e.m.) peak dopamine across all value experiments (n = 4 rats). e, Average (± s.e.m.) VMS dopamine during T-maze (n = 44 sessions in 3 rats, black) and M-maze (n = 17, blue) sessions in same rats. f, g, Average (± s.e.m.) peak dopamine signals for the sessions plotted in a (f) and b (g) for trials to left (blue) and right (red) goals. Shading indicates arm with larger reward. h, i, Average normalized dopamine (h) and running speed (i) for runs to high (light green) and low (dark green) reward goals in the M-maze. Vertical lines indicate turns. j, k, Average normalized dopamine (j) and running speed (k) in the S-maze (n = 9 sessions in 2 rats), plotted as in h and i.

Figure 4. Ramping dopamine selectivity can emerge with training without experimentally imposed reward discrepancies

a, Average normalized dopamine at a VMS site as a function of maze location (n = 19 sessions). b, Dopamine selectivity indices (Methods) for all individual sessions averaged in a. c, Average running speed for sessions in a. d, Selectivity indices for all VMS (left) and DLS (right) recordings (red) compared to shuffled data (blue) for all rats (n = 9). e, f, Average percent correct performance (e) and average Z-score normalized dopamine selectivity (f) across training blocks. Error bars, s.e.m.