Individual differences in reinforcement learning: behavioral, electrophysiological, and neuroimaging correlates

Abstract

During reinforcement learning, phasic modulations of activity in midbrain dopamine neurons are conveyed to the dorsal anterior cingulate cortex (dACC) and basal ganglia (BG) and serve to guide adaptive responding. While the animal literature supports a role for the dACC in integrating reward history over time, most human electrophysiological studies of dACC function have focused on responses to single positive and negative outcomes. The present electrophysiological study investigated the role of the dACC in probabilistic reward learning in healthy subjects using a task that required integration of reinforcement history over time. We recorded the feedback-related negativity (FRN) to reward feedback in subjects who developed a response bias toward a more frequently rewarded ("rich") stimulus ("learners") versus subjects who did not ("non-learners"). Compared to non-learners, learners showed more positive (i.e., smaller) FRNs and greater dACC activation upon receiving reward for correct identification of the rich stimulus. In addition, dACC activation and a bias to select the rich stimulus were positively correlated. The same participants also completed a monetary incentive delay (MID) task administered during functional magnetic resonance imaging. Compared to non-learners, learners displayed stronger BG responses to reward in the MID task. These findings raise the possibility that learners in the probabilistic reinforcement task were characterized by stronger dACC and BG responses to rewarding outcomes. Furthermore, these results highlight the importance of the dACC to probabilistic reward learning in humans.

Fig. 1

(A) Averaged ERP waveforms at Fz and FCz from 200 ms before to 600 ms after the presentation of reward feedback for the rich stimulus during the probabilistic reward task for learners (light line) and non-learners (heavy line) in the probabilistic reward task; and (B) amplitude of the FRN at Fz during early (block 1) and late phases (blocks 2 & 3) of learning. Error bars refer to standard errors.

Fig. 2

Results of voxel-by-voxel independent t-tests contrasting current density for the learners and non-learners in response to reward feedback for the rich stimulus on the probabilistic reward task. Red: relatively higher activity for learners. Blue: relatively higher activity for non-learners. Statistical map is thresholded at p<0.020 (minimum cluster size: 5 voxels) and displayed on the MNI template.

Fig. 3

Scatterplot and Pearson correlation between increases in dACC activation and response bias from early (block 1) to late phases (blocks 2 & 3) of learning. Relatively increased dACC current density in response to reward feedback for the rich stimulus is associated with greater reward learning (r=0.40, p<0.030). When the subject with the lowest reward learning was omitted, the correlation was r=0.59, p<0.001.

Fig. 4

(A) Parcellation of basal ganglia structures in a representative participant; only the caudate, putamen, and globus pallidus are shown in this coronal slice. (B) Mean beta weights (averaged across regions and hemispheres) in response to win feedback and no-win feedback in learners and non-learners (significant Group × Outcome interaction). Error bars refer to standard errors.