Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making

Abstract

Dual system theories suggest that behavioral control is parsed between a deliberative "model-based" and a more reflexive "model-free" system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [(18)F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.

Keywords: PET; decision making; dopamine; fMRI; reinforcement learning.

Fig. 1.

Behavioral task and relation to presynaptic dopamine. (A) Exemplary trial sequence of the two-step decision task and timing. (B) Illustration of the state transition matrix. (C) Mean voxelwise K_i map of 29 participants and borders of striatal regions of interest. (D) Correlation between right ventral striatal K_i and the balance of model-free and model-based choices ω (r = 0.31; P = 0.04) and between right ventral striatal K_i and the reaction times for common versus rare states (r = 0.38; P = 0.04).

Fig. 2.

fMRI results. Model-free prediction errors (Left), additional model-based signals (Middle), and the conjunction of both (Right) in ventral striatum (VS, Upper) and lateral prefrontal cortex (lPFC, Lower). For display purposes, all statistical maps are thresholded at a minimum T value of 3.24 (corresponding to P < 0.001, uncorrected) with a cluster extent k = 20. For details, see Table S3.

Fig. 3.

Presynaptic dopamine and neural learning signatures. Correlation between right ventral striatal presynaptic dopamine K_i and (A) model-free learning signals in right ventral striatum (r = −0.37; P = 0.02) and (B) model-based signatures in right lateral prefrontal cortex (r = 0.38; P = 0.03).