Striatal circuits for reward learning and decision-making

Abstract

The striatum is essential for learning which actions lead to reward and for implementing those actions. Decades of experimental and theoretical work have led to several influential theories and hypotheses about how the striatal circuit mediates these functions. However, owing to technical limitations, testing these hypotheses rigorously has been difficult. In this Review, we briefly describe some of the classic ideas of striatal function. We then review recent studies in rodents that take advantage of optical and genetic methods to test these classic ideas by recording and manipulating identified cell types within the circuit. This new body of work has provided experimental support of some longstanding ideas about the striatal circuit and has uncovered critical aspects of the classic view that are incorrect or incomplete.

Fig. 1 |. Heterogeneity of midbrain dopamine neurons.

a| Schematic of the organization of projections from the substantia nigra pars compacta (SNc) and ventral tegmental area (VTA) to the striatum. The VTA predominantly projects to the ventral striatum and the SNc to the dorsal striatum^–. A distinct population of dopamine (DA) neurons in the lateral SNc project to the tail of the striatum (TS). b | Schematized examples of the heterogeneous activity profiles of DA neurons and their organization in the SNc and VTA. Reward prediction error signals are found throughout VTA and SNc neurons^,,. DA neurons increase their activity in response to both unexpected rewards (left) or cues (such as an auditory tone) that predict reward. In the medial VTA, some DA neurons signal the accuracy of the trial. The activity of other VTA neurons is correlated with the distance of the animal from a reward^,. Activity in some DA neurons in the lateral VTA and SNc is associated with movement^,,,. Activity of some SNc DA axons in the striatum is more correlated with salience than with value, in that they respond similarly to unexpected positive events (for example, a water reward) and negative events (such as a foot shock). Neurons in the lateral SNc show weak responses to reward and respond more robustly to salient or threatening stimuli (such as an air puff)^,. DLS, dorsolateral striatum; DMS, dorsomedial striatum; NAcc, nucleus accumbens core; NAcsh, nucleus accumbens shell.

Fig. 2 |. Direct and indirect pathway regulation of behaviour.

a | Simplified schematic of the direct and indirect pathways through the basal ganglia. The ‘go/no-go’ model of direct pathway and indirect pathway function proposes that when the D₁ dopamine receptor (D1R) medium spiny neurons (MSNs) that make up the direct pathway are activated (left), they inhibit the primary output nuclei of the basal ganglia: the internal globus pallidus (GPi) and the substantia nigra pars reticulata (SNr). The GPi and SNr tonically inhibit brainstem and thalamic nuclei, which become disinhibited via D1R MSN activation. When the dopamine D₂ receptor (D2R) MSNs of the indirect pathway are activated (right), they inhibit the external globus pallidus (GPe), which sends inhibitory projections to the GPi, SNr and subthalamic nucleus (STN). Thus, the GPi, SNr and STN are disinhibited. The STN sends excitatory input to the GPi and SNr, further activating it and inhibiting their brainstem and thalamic targets^,,,,,,. b | Schematized data from cell-type-specific extracellular recordings. D1R and D2R neurons have similar activation patterns and selectivity during chosen actions but oppositely encode outcome. c | In a decision-making task, optogenetic stimulation of D1R or D2R MSNs induces opposite biases that depend on the difference in action value (the estimated value of the outcome resulting from an action) between the two options. Part b is adapted with permission from reF., Elsevier. Part c is adapted with from reF., Springer Nature Limited.

Fig. 3 |. Cholinergic interneurons modulate synaptic plasticity and cocaine context extinction learning.

a | Simplified schematic of the striatal circuit, highlighting the cholinergic interneurons (CINs) as well as D₁ (D1R) and D₂ dopamine receptor (D2R) medium spiny neurons (MSNs). All cell types receive glutamatergic and dopaminergic inputs from external structures. b | A recent study manipulated CINs during the extinction of a cocaine-context association. Increased CIN activity was associated with increased presynaptic plasticity (corresponding to reduced glutamate release) and increased extinction learning. In addition, extended extinction training in control animals was associated with a similar change in synaptic plasticity, suggesting that CINs accelerate the plasticity associated with extinction learning. SNc, substantia nigra pars compacta; VTA, ventral tegmental area. Part b is adapted with permission from reF., Elsevier.

Fig. 4 |. Glutamatergic inputs to the striatum.

a | Schematic of three example cortico–basal ganglia–thalamo–cortical loops with corticostriatal projections from the prelimbic cortex (PL), anterior cingulate cortex (ACC) and primary motor cortex (M1). Inputs to the striatum are topographically organized, and this organization persists throughout the basal ganglia^,,. b | Glutamatergic inputs provide functional specialization to striatal subregions. For example, the projection from the auditory cortex (AC) to the tail of the striatum (TS) is tonotopically organized, and frequency tuning of TS neurons corresponds to that of their AC inputs. When rats learn to perform a right or left nose poke in response to a low-frequency or high-frequency auditory stimulus to obtain reward (upper left), corticostriatal synapses tuned to the rewarded frequency are selectively potentiated (indicated in red, upper right). Thus, if mice learn to perform a right nose poke following a low-frequency stimulus, inputs from the left (that is, contralateral) AC to the left striatum that are tuned to low frequencies will be strengthened (upper right). In the same task, optogenetic manipulation of neurons projecting to the TS from the AC bidirectionally biases choice (lower). Activation of these neurons biases choice towards, whereas inhibition biases choice away from, the preferred frequency of the manipulated AC neurons. DLS, dorsolateral striatum; DMS, dorsomedial striatum; GP, globus pallidus; MD, medial dorsal thalamus; NAc, nucleus accumbens; VAL, ventral anterior-lateral complex; VM, ventromedial nucleus; VP, ventral pallidum. Part b is adapted from reF., Springer Nature Limited.