Stimulus-response and response-outcome learning mechanisms in the striatum

Abstract

While midbrain DA neurons show phasic activations in response to both reward-predicting and salient non-reward events, activation responses to primary and conditioned rewards are sustained for several hundreds of milliseconds beyond those elicited by salient non-reward-related stimuli. The longer-duration DA reward response and corresponding elevated DA release in striatal target sites may selectively strengthen currently-active corticostriatal synapses, i.e., those associated with the successful reward-procuring behavior. This paper describes how similar models of DA-mediated plasticity of corticostriatal synapses may describe both stimulus-response and response-outcome learning. DA-mediated strengthening of corticostriatal synapses in regions of the dorsolateral striatum receiving afferents from primary sensorimotor cortex is likely to bind corticostriatal inputs representing the previously-emitted movement to striatal outputs contributing to the selection of the next movement segment in a behavioral sequence. Within the striatum, more generally, inputs from distinct regions of the frontal cortex that code independently for movement direction and reward expectation send convergent projections to striatal output cells. DA-mediated strengthening of active corticostriatal synapses promotes the future output of the striatal cell under similar input conditions. This is postulated to promote persistence of neuronal activity in the very cortical cells that drive corticostriatal input, leading to the establishment of sustained reverberatory loops that permit cortical movement-related cells to maintain activity until the appropriate time of movement initiation.

Figure 1

Figure 1A. Corticostriatal inputs (S_A, S_B) coding for sensory events synapse upon striatal output cells coding for behavioral responses (R_A, R_B). A behavioral response (R_A) leading to delivery of a reward or reward-predicting stimulus causes a phasic increase in midbrain DA activity and a consequent increase in striatal DA release. DA-mediated LTP strengthens the currently-active synapses (S_B to R_A; bottom diagram, bold box in striatum) and increases the likelihood that the reward-procuring behavioral response (R_A) will occur in response to the same sensory stimulus (S_B) in the future. If a striatal output cell is activated by convergent inputs from cortical cells representing two sensory modalities, DA activation will increase the synaptic strength of both inputs to the striatal cell. Such a cell will come to show preferential activity in response to the conjunction of the two sensory conditions, e.g., to a visual stimulus only when presented in proximity to a particular tactile field [55]. Figure 1B. Corticostriatal inputs (O_A, O_B) coding for expected reward outcomes synapse upon striatal output cells coding for behavioral responses (R_A, R_B). Selective strengthening of O-R synapses occurs in a manner identical to that depicted for S-R synapses in figure 1A.

Figure 1

Figure 1A. Corticostriatal inputs (S_A, S_B) coding for sensory events synapse upon striatal output cells coding for behavioral responses (R_A, R_B). A behavioral response (R_A) leading to delivery of a reward or reward-predicting stimulus causes a phasic increase in midbrain DA activity and a consequent increase in striatal DA release. DA-mediated LTP strengthens the currently-active synapses (S_B to R_A; bottom diagram, bold box in striatum) and increases the likelihood that the reward-procuring behavioral response (R_A) will occur in response to the same sensory stimulus (S_B) in the future. If a striatal output cell is activated by convergent inputs from cortical cells representing two sensory modalities, DA activation will increase the synaptic strength of both inputs to the striatal cell. Such a cell will come to show preferential activity in response to the conjunction of the two sensory conditions, e.g., to a visual stimulus only when presented in proximity to a particular tactile field [55]. Figure 1B. Corticostriatal inputs (O_A, O_B) coding for expected reward outcomes synapse upon striatal output cells coding for behavioral responses (R_A, R_B). Selective strengthening of O-R synapses occurs in a manner identical to that depicted for S-R synapses in figure 1A.

Figure 2

Corticostriatal inputs from primary sensorimotor cortex carry somatosensory (S_B) and movement-related (R_1A) inputs representing the just-initiated behavior to striatal output cells corresponding with the to-be-initiated movement (one of the two R_2A cells). If the animal initiates a behavioral response leading to reward delivery, the currently active synapses (S_B/R_1A to R_2A) are strengthened, increasing the likelihood that the reward-procuring behavior (R_2A) will occur in response to the same somatosensory and motor input conditions (S_B/R_1A) in the future.

Figure 3

Cortical M1 and SMA inputs coding the direction of an upcoming limb movement (R_A, R_B) and cortical cells coding expected reward outcome (O_A, O_B) send converging projections to striatal output cells (a, b, c, d) which, via striato-pallido-thalamo-cortical pathways promote persistence of activity in the cortical cells. If the animal initiates a behavioral response leading to a reward or reward-predicting cue, the currently active synapses (R_A/O_B to b) are strengthened by DA-mediated LTP, increasing the likelihood that the same cortical inputs will produce reverberatory activity in the circuit in the future. Once the relevant corticostriatal synapses have been strengthened by DA responses to the reward-predicting cue (e.g., a trigger stimulus associated with response-contingent reward delivery), the likelihood of future reverberatory activity through those synapses will be increased even if future inputs arrive earlier in the trial (e.g., seconds prior to the trigger stimulus).