A process model account of the role of dopamine in intertemporal choice

Abstract

Theoretical accounts disagree on the role of dopamine in intertemporal choice and assume that dopamine either promotes delay of gratification by increasing the preference for larger rewards or that dopamine reduces patience by enhancing the sensitivity to waiting costs. Here, we reconcile these conflicting accounts by providing empirical support for a novel process model according to which dopamine contributes to two dissociable components of the decision process, evidence accumulation and starting bias. We re-analyzed a previously published data set where intertemporal decisions were made either under the D2 antagonist amisulpride or under placebo by fitting a hierarchical drift diffusion model that distinguishes between dopaminergic effects on the speed of evidence accumulation and the starting point of the accumulation process. Blocking dopaminergic neurotransmission not only strengthened the sensitivity to whether a reward is perceived as worth the delay costs during evidence accumulation (drift rate) but also attenuated the impact of waiting costs on the starting point of the evidence accumulation process (bias). In contrast, re-analyzing data from a D1 agonist study provided no evidence for a causal involvement of D1R activation in intertemporal choices. Taken together, our findings support a novel, process-based account of the role of dopamine for cost-benefit decision making, highlight the potential benefits of process-informed analyses, and advance our understanding of dopaminergic contributions to decision making.

Trial registration: ClinicalTrials.gov NCT03181841.

Keywords: amisulpride; delay discounting; dopamine; drift diffusion model; human; neuroscience; proximity; reward.

Figure 1.. Task design and experimental procedures.

(A) Participants made choices between alternatives that provided smaller-sooner rewards (e.g., 100 Swiss francs in 0 day) or larger-later rewards (e.g., 250 Swiss francs in 60 days). (B) In a double-blind crossover design, participants performed the intertemporal decision task after administration of the D2 antagonist amisulpride or placebo on two separate days. (C) Model-free Bayesian analyses revealed weaker influences of delay costs on decision making under amisulpride compared with placebo, consistent with previous findings that D2R antagonism strengthens the preference for delayed rewards. (D) Individual coefficients for the impact of delay on choices in the amisulpride and placebo conditions. (E) Illustration of the choice process in the framework of a drift-diffusion model. After a non-decision time τ (not shown here), evidence is accumulated from a starting point ζ with the weighted difference between benefits and action costs determining the speed of the accumulation process (drift rate v) toward the boundaries for the larger-later or smaller-sooner option. (F) Delay discounting under placebo (log-k; dots correspond to individual participants, with more negative values indicating weaker delay discounting) decreased with the difference in weights assigned to rewards and delay costs during evidence accumulation, replicating previous findings (Amasino et al., 2019).

Figure 2.. D2R blockade affects multiple components of the intertemporal decision process.

(A) Larger differences in reward magnitude between the larger-later (LL) and smaller-sooner (SS) option increased drift rates, speeding up evidence accumulation toward LL options under placebo. (B) The impact of differences in reward magnitude was significantly stronger under amisulpride than under placebo. (C) Drug-dependent impact of differences in reward magnitude on the drift rate. Because the sensitivity to differences in reward magnitude was stronger under amisulpride than under placebo (steeper slope), D2R blockade sped up evidence accumulation toward the boundary for LL choices if differences in reward magnitude between the LL and SS options were large. In contrast, if the difference in reward magnitude was small, the drift rate was more negative under amisulpride compared with placebo, speeding up evidence accumulation toward the SS option. (D) Larger differences between delay of reward promoted evidence accumulation toward the (negative) boundary for SS choices under placebo, (E, F) but the impact of delay was not significantly altered by amisulpride. (G) The starting point of the accumulation processes was closer to the boundary for LL than SS choices under placebo, (H) and this starting bias toward the LL option was significantly reduced by amisulpride. (I) For larger differences in waiting costs, the starting point of the evidence accumulation process was increasingly shifted toward the SS option under placebo. (J) This impact of delay costs on the starting bias was significantly reduced under amisulpride. As illustrated in (K), reducing dopaminergic action on D2R with amisulpride shifted the starting bias toward the boundary for the SS option predominantly if no option possessed a clear proximity advantage (small difference between delays). In A, B, D, E, G, H, I, and J, yellow bars close to x-axis indicate 95% HDIs.

Figure 3.. Model comparison.

(A) The deviance information criterion (DIC; lower numbers correspond to better fit) suggests that DDM-1 explained the data slightly better than DDM-2, DDM-4, and DDM-6 and clearly outperformed DDM-3 and DDM-5. (B, C) Posterior predictive checks on the group level (collapsed across all participants), separately for (B) placebo and (C) amisulpride. Particularly DDM-1, DDM-2, and DDM-3 described the empirically observed data well, whereas decisions simulated based on DDMs 4–6 more strongly deviated from observed behavior.

Figure 4.. Posterior predictive checks.

For each individual participant (p1–p56), observed RTs (in black) are plotted against the RTs simulated based on the parameters for drift diffusion model (DDM) 1–6, separately for differences in (A) reward magnitude and (B) delay (quintiles). The plots suggest that the DDMs provide reasonable accounts of the observed RTs.

Figure 5.. Illustration of how dopaminergic effects on intertemporal choices depend on differences in both reward magnitude and delay in the proposed framework, separately for (A) placebo, (B) amisulpride, and (C) the difference between amisulpride and placebo.

Plots are based on simulations assuming the group-level parameter estimates we observed under placebo and amisulpride. As dopaminergic effects on decision making affect both reward processing (via the drift rate) and cost processing (via the starting bias), the specific combination of rewards and delays determines whether D2R blockade increases or decreases the probability of larger-later (LL) choices. Low dopamine levels reduce the proximity advantage of smaller-sooner (SS) over LL options particularly if differences in action costs between reward options are large, promoting choices of the LL option. In contrast, if no option possesses a proximity advantage (small differences between delays), dopaminergic effects on evidence accumulation dominate, such that the LL option is perceived as less worth the waiting costs, particularly if its reward magnitude differs only little from that of the alternative SS option.