The role of ventral frontostriatal circuitry in reward-based learning in humans

Abstract

This study examined changes in behavior and neural activity with reward learning. Using an event-related functional magnetic resonance imaging paradigm, we show that the nucleus accumbens, thalamus, and orbital frontal cortex are each sensitive to reward magnitude, with the accumbens showing the greatest discrimination between reward values. Mean reaction times were significantly faster to cues predicting the greatest reward and slower to cues predicting the smallest reward. This behavioral change over the course of the experiment was paralleled by a shift in peak in accumbens activity from anticipation of the reward (immediately after the response), to the cue predicting the reward. The orbitofrontal and thalamic regions peaked in anticipation of the reward throughout the experiment. Our findings suggest discrete functions of regions within basal ganglia thalamocortical circuitry in adjusting behavior to maximize reward.

Figure 1.

Task design. One of three cues (cartoon pirate) appeared on the left or right side of a fixation for 1 s. After a 2 s delay, a response prompt (2 treasure chests) appeared for 2 s, and subjects were instructed to press with their pointer finger if the cue had been on the left and with their middle finger if the cue had been on the right. After another 2 s delay, a reward outcome (small, medium, or large pile of coins) was presented and was followed by a 12 s ITI. The total trial length was 20 s.

Figure 2.

Behavioral results. There was a main effect of reward and a significant interaction of time on task (learning) by reward. After learning, subjects were faster when responding to cues associated with large (348 ± 74 ms) rewards (•) relative to the medium (501 ± 160 ms) rewards (▪) and slower to small (576 ± 176 ms) rewards (▴). Runs (time on task) are on the x-axis, and reaction time (in seconds) is on the y-axis.

Figure 3.

Nucleus accumbens percentage MR signal change during early (a), middle (b), and late (c) trials for small, medium, and large rewards after cue presentation. Time is on the x-axis, and the percentage of MR signal change is on the y-axis. Plots are not adjusted for the hemodynamic response.

Figure 4.

Left, Activation in the accumbens (x = 8, y = 6, z = –2, and x = –8, y = 6, z = –5) was greater for the large relative to the small cue (based on t tests between the β weights of the small vs large cue predictors). Right, The percentage of the MR signal change for the large reward magnitude for early versus late trials in the accumbens region during cue presentation. Error bars indicate SE.

Figure 5.

Activation in the medial thalamus (a) (x = 7, y = –21, z = 9) and orbital frontal cortex (b) (x = 45, y = 48, z = –2) was greatest in anticipation of the reward (after the rewarded response) during late trials, but neither the thalamus (c) nor the orbital frontal cortex (d) discriminated between medium and large rewards. Time is on the x-axis, and the percentage of MR signal change is on the y-axis.

Figure 6.

Temporal dissociation of basal ganglia thalamocortical regions. The accumbens MR signal onset occurs first, followed by the thalamus and orbital frontal cortex. Time is on the x-axis, and the percentage of MR signal change is on the y-axis.