Role of dopamine D2 receptors in human reinforcement learning

Abstract

Influential neurocomputational models emphasize dopamine (DA) as an electrophysiological and neurochemical correlate of reinforcement learning. However, evidence of a specific causal role of DA receptors in learning has been less forthcoming, especially in humans. Here we combine, in a between-subjects design, administration of a high dose of the selective DA D2/3-receptor antagonist sulpiride with genetic analysis of the DA D2 receptor in a behavioral study of reinforcement learning in a sample of 78 healthy male volunteers. In contrast to predictions of prevailing models emphasizing DA's pivotal role in learning via prediction errors, we found that sulpiride did not disrupt learning, but rather induced profound impairments in choice performance. The disruption was selective for stimuli indicating reward, whereas loss avoidance performance was unaffected. Effects were driven by volunteers with higher serum levels of the drug, and in those with genetically determined lower density of striatal DA D2 receptors. This is the clearest demonstration to date for a causal modulatory role of the DA D2 receptor in choice performance that might be distinct from learning. Our findings challenge current reward prediction error models of reinforcement learning, and suggest that classical animal models emphasizing a role of postsynaptic DA D2 receptors in motivational aspects of reinforcement learning may apply to humans as well.

Figure 1

Sulpiride effects on reinforcement learning. The learning curves depict the ratio of volunteers that chose the ‘correct' stimulus in the gain domain (upper graph, solid lines), and the ‘incorrect' stimulus in the loss domain (lower graph, dashed lines). In the gain domain, the ‘correct' stimulus is associated with a probability of 0.75 of winning £1, whereas in the loss domain the ‘incorrect' stimulus is associated with a probability of 0.75 of losing £1. (a) Sulpiride (red) compared with placebo (blue) induces behavioral impairments in the gain domain after learning has plateaued (Mann–Whitney test, p=0.016, n=76). (b) Sulpiride group divided by serum values through median split. Higher serum levels (red) relate to more prominent impairments in the gain domain when compared with volunteers with lower serum levels (blue, Mann–Whitney test, p=0.031, n=39).

Figure 2

Parameter estimates of the Q-learning model across drug, serum value and genotype groups, separately for the gain and loss domain. (a) Temperature parameter β_gain is significantly higher in the sulpiride group (57% increase compared with the placebo, p=0.005), but the learning rate α_gain is not affected, and there are no effects in the loss domain (α_loss, β_loss). (b) Higher sulpiride serum values selectively affect the temperature parameter β_gain (183% increase in high compared with low serum values, p=0.001), with no effects on either α_gain, α_loss or β_loss. (c) Pronounced sulpiride effects on β_gain are observed in A1+ genotype carriers (211% increase following sulpiride compared with placebo administration, p<0.001), but not in A1− genotype carriers.

Figure 3

Response latencies over trials. (a) Whereas response latencies decrease over trials in both groups, volunteers in the sulpiride (red) compared with the placebo (blue) group show higher response latencies in the gain domain in the plateau phase of learning (t-test, p=0.044, n=76). (b) Response latencies in the loss domain are higher than in the gain domain, and there is no significant difference between the sulpiride group (red) and the placebo group in the loss domain (blue).

Figure 4

Pharmacogenetic interaction of sulpiride and DA D2 receptor Taq1A polymorphism on reinforcement learning. (a) Volunteers with a genetically determined reduction in DA D2 receptor density (A1+ allele carriers) show behavioral impairments during reinforcement learning following sulpiride (red) compared with placebo (blue) administration in the gain domain (upper graph, solid lines, Mann–Whitney test, p=0.048, n=38), but not in the loss domain (lower graph, dashed lines). (b) Sulpiride (red) compared with placebo (blue) has no significant effect in either the gain (upper graph, solid lines) or the loss (lower graph, dashed lines) domain in volunteers who carry the common variant (A1−) of the polymorphism.