Neural determinants of human goal-directed vs. habitual action control and their relation to trait motivation

Abstract

Instrumental learning is mediated by goal-directed and habit systems in the brain. While rodent studies implicate distinct prefrontal/striatal regions in goal-directed and habit learning, neural systems underpinning these two processes in humans remain poorly understood. Here, using a validated discrimination learning task that distinguishes goal-directed learning from habit learning in 72 subjects in fMRI, we investigated the corticostriatal correlates of goal-directed learning and tested whether brain activation during learning is associated with trait motivation and behavioral performance in the post-learning test phase. Participants showed enhanced activation in medial prefrontal and posterior cingulate cortices during goal-directed action selection in the training phase, whereas habitual action selection activated bilateral insula, bilateral dorsal caudate and left precentral gyrus. In addition, early phase of learning was associated with increased activation in the frontoparietal control network and dorsal striatum, whereas default mode regions depicted increased activation in the late phase. Finally, avoidance motivation scores measured by Behavioral Inhibition/Activation System (BIS/BAS) correlated with accuracy during goal-directed learning and showed a nominally significant correlation with activation in dorsomedial prefrontal cortex during goal-directed acquisition of stimuli. These findings reveal the temporal dynamics of instrumental behavior and suggest that avoidance motivation predicts performance and brain activity during goal-directed learning.

Figure 1

Experimental paradigm. (A) The Discrimination Learning Task performed in the scanner is shown. In the illustrated example, subjects are presented with a closed box and a banana picture (cue) in front of it. They are asked to press the key associated with the banana, which they learn via feedback presented after they make their choice. If the subject presses the correct key, then the box opens with another fruit inside and a number showing the points earned. If the wrong key is pressed or response is omitted, an empty box opens. In the standard condition, a given fruit is only used either as cue or as outcome (but not as both), whereas in the incongruent condition, a given fruit is used as cue in some trials and as outcome in others. The task also includes a control condition, in which an arrow is presented above the box instead of a fruit and shows the direction of the key to be pressed. The box then opens with a blue circle displayed above it. (B) The Outcome Devaluation Test is shown. In this example trial, two open boxes are presented. The “X” on kiwi indicates that this fruit is devalued and no longer offers points. Therefore, the correct response is the right key, which led participants to the valuable outcome (blackberry) during the learning phase. (C) The Slip Test is illustrated. At the beginning of each block, subjects are presented with outcome fruits to obtain in the upcoming set of trials. Unlike the learning task, 2 fruits (out of 6 in a block) are devalued and collecting them would result in subtraction of points. Those devalued outcomes are marked with ‘X’s. In the example shown, kiwi is an outcome to avoid, therefore, when presented with a banana, subjects should not press any key to avoid loss of points. The subjects are asked to collect the valuable fruits. Since the pear leads to a valuable fruit (blackberry), the subjects should press the correct key (right) in response to pear. The fruit images in this figure were obtained from the food-pics database for illustration purpose and their brightness was adapted for this figure. The license agreement is available on https://creativecommons.org/licenses/by/3.0/legalcode.

Figure 2

Behavioral results. (A) Subjects’ accuracy and reaction time are displayed for the learning phase. Subjects rapidly learned both discriminations and reached high levels of accuracy at the end of six blocks. Their responses gradually became faster in both conditions, and overall they were significantly faster in the standard condition. (B) In the Outcome Devaluation Test, subjects’ responses to outcomes from standard pairs were more accurate (than incongruent). (C) Percentage of the responses made during the Slip Test is shown for valuable/devalued outcomes in both conditions. Subjects responded less often when the outcomes were devalued in both conditions, however, there was a significant interaction of Value X Discrimination, i.e., subjects responded for devalued outcomes more often and responded for valuable outcomes less often during incongruent trials. Error bars represent the standard error of the mean. Abbreviations: RT: reaction time, ODT: Outcome Devaluation Test, ST: Slip Test.

Figure 3

Brain activation during cue and outcome presentation. (A) Brain regions that were more active during standard and incongruent cues are depicted. PCC and medial prefrontal areas demonstrated greater activation during standard cues, whereas bilateral insula, dorsal caudate and left precentral gyrus showed enhanced activation during incongruent cues. (B) No region reached significance for Standard > Incongruent outcomes, however, left SPL and right inferior frontal gyrus demonstrated stronger activation during incongruent outcomes. All clusters displayed on the maps survive correction for multiple comparisons using a cluster-defining threshold of p < 0.001 and 10,000 Monte Carlo simulations at clusterwise p < 0.05. Abbreviations: dmPFC: dorsomedial prefrontal cortex, IFG: inferior frontal gyrus, PCC: posterior cingulate cortex, PrCG: precentral gyrus, SPL: superior parietal lobule, vmPFC: ventral medial prefrontal cortex.

Figure 4

Brain activation during early and late learning phases. (A) Regions depicting stronger activation in early (first 2 blocks) versus late phases (last 2 blocks) of learning. Stronger activation was observed in bilateral dorsal caudate, bilateral insula, dmPFC, and right dlPFC during cue presentation in the early phase in both standard and incongruent trials. Conversely, in the late phase, PCC, bilateral postcentral gyrus, and right superior temporal gyrus showed enhanced activation only during standard trials. During outcome presentation, stronger activation was observed in dACC and right dlPFC in the early phase, whereas ventral striatum activation was enhanced in the late phase. All clusters displayed on the maps survive correction for multiple comparisons using a cluster-defining threshold of p < 0.001 and 10,000 Monte Carlo simulations at clusterwise p < 0.05. Abbreviations: dACC: dorsal anterior cingulate cortex, dlPFC: dorsolateral prefrontal cortex, dmPFC: dorsomedial prefrontal cortex, PCC: posterior cingulate cortex.