Corticoinsular circuits encode subjective value expectation and violation for effortful goal-directed behavior

Abstract

We are presented with choices each day about how to invest our effort to achieve our goals. Critically, these decisions must frequently be made under conditions of incomplete information, where either the effort required or possible reward to be gained is uncertain. Such choices therefore require the development of potential value estimates to guide effortful goal-directed behavior. To date, however, the neural mechanisms for this expectation process are unknown. Here, we used computational fMRI during an effort-based decision-making task where trial-wise information about effort costs and reward magnitudes was presented separately over time, thereby allowing us to model distinct effort/reward computations as choice-relevant information unfolded. We found that ventromedial prefrontal cortex (vmPFC) encoded expected subjective value. Further, activity in dorsal anterior cingulate (dACC) and anterior insula (aI) reflected both effort discounting as well as a subjective value prediction error signal derived from trial history. While prior studies have identified these regions as being involved in effort-based decision making, these data demonstrate their specific role in the formation and maintenance of subjective value estimates as relevant information becomes available.

Keywords: anterior insula; dorsal anterior cingulate; effort-based decision making; prediction error; ventromedial prefrontal cortex.

Fig. 1.

(A) Schematic of experimental task design. This image shows the time line of a trial in which effort and reward information is presented sequentially. This is an example of an Effort First trial. Each trial began with the presentation of a fixation cross, followed by the Cue 1 phase in which one piece of information was presented (either effort level or reward magnitude). After 2–6 s, the second piece of information was presented (Cue 2). After an additional 2–6 s, participants saw a Decision Prompt that prompted them to make a choice between the Effort Option presented and the No Effort Option that always paid $1.00. They were required to make their selection within 3 s. Following their selection, their choice was presented to them during the Choice Phase. (B) Proportion of effortful choices based upon effort level and reward magnitude. Participants chose more effortful options as reward increased and as effort decreased. Error bars are all SEM. (C) Individual and group average (dark blue) subjective value curves based on the results of our computational model. The group average is shown as the dark blue line with shading around it that represents the SE. The remaining colored lines each reflect a single participant, demonstrating individual differences in discounting.

Fig. 2.

(A) Increased BOLD signal in dACC, putamen, insula, and supplementary motor area at Cue 2. (B) Increased BOLD signal in dACC in response to decreasing subjective value. The effect size plot demonstrates the negative relationship between BOLD activity and subjective value magnitude in dACC. (C) Increased BOLD signal in vmPFC in response to subjective value magnitude. The effect size plot demonstrates the positive relationship between BOLD activity and subjective value magnitude in vmPFC.

Fig. 3.

(A) Increased percent signal change (PSC) in aI to high-effort information at Cue 2 when presented following high reward at Cue 1 compared with low effort. (B) No observable PSC response to high effort when presented at Cue 1 regardless of reward level presented at Cue 2.

Fig. 4.

(A) Increased BOLD activity in bilateral aI in response to unsigned SVPE generation. The effect size plot demonstrates this positive relationship between BOLD signal and SVPE. (B) BOLD activity in insula is significantly greater in response to SVPE than to subjective value. Further, within prediction error, dorsal insula activity is significantly stronger than ventral insula activity. (C) Increased BOLD activity in dACC in response to unsigned SVPE. The effect size plot demonstrates this positive relationship between BOLD signal and prediction error encoding. (D) BOLD activity in anterior dACC is significantly stronger than in posterior dACC for subjective value. In posterior dACC, greater BOLD activity was observed in response to SVPE than to subjective value alone. (E) Increased BOLD activity in caudate in response to unsigned SVPE. The effect size plot demonstrates this positive relationship between BOLD signal and prediction error encoding. (F) BOLD activity in caudate is significantly greater in response to SVPE than to subjective value. Further, posterior caudate is more active than anterior caudate for prediction error encoding. *P < 0.05; **P < 0.005; ***P < 0.001.

Fig. 5.

Increased BOLD activity at Cue 1 in response to expected subjective value when reward information is presented first in vmPFC. The effect size plot illustrates a positive relationship between BOLD signal and predicted reward magnitude.