Reward-related activity in ventral striatum is action contingent and modulated by behavioral relevance

Abstract

Multiple features of the environment are often imbued with motivational significance, and the relative importance of these can change across contexts. The ability to flexibly adjust evaluative processes so that currently important features of the environment alone drive behavior is critical to adaptive routines. We know relatively little about the neural mechanisms involved, including whether motivationally significant features are obligatorily evaluated or whether current relevance gates access to value-sensitive regions. We addressed these questions using functional magnetic resonance imaging data and a task design where human subjects had to choose whether to accept or reject an offer indicated by visual and auditory stimuli. By manipulating, on a trial-by-trial basis, which stimulus determined the value of the offer, we show choice activity in the ventral striatum solely reflects the value of the currently relevant stimulus, consistent with a model wherein behavioral relevance modulates the impact of sensory stimuli on value processing. Choice outcome signals in this same region covaried positively with wins on accept trials, and negatively with wins on reject trials, consistent with striatal activity at feedback reflecting correctness of response rather than reward processing per se. We conclude that ventral striatum activity during decision making is dynamically modulated by behavioral context, indexed here by task relevance and action selection.

Keywords: action value; multisensory; policy selection; reward; ventral striatum.

Figure 1.

A, Diagram illustrating different models for the effect of task relevance on value processing. Task relevance can modulate the input of sensory signals (S1–S3) to value-processing regions, leading to a single relevance-modulated value signal (V) that is then used for action selection and planning (A; left). Alternatively, task relevance can modulate the output of value-processing regions, which predicts the simultaneous representation of the value of all motivationally valenced stimulus features (right). B, Activity in bilateral ventral striatum reflects the value of behaviorally relevant features of the environment (left) but not behaviorally irrelevant ones (right). The difference between these contrasts (the interaction effect) was significant in both hemispheres. This supports models where task relevance modulates inputs to value-processing regions, leading to the generation of a single relevance-modulated value prediction, as illustrated in the diagram on the left side of Figure 2A. (Image thresholded at p < 0.005 uncorrected for display purposes. Color bar indicates the voxelwise T statistic; Y = 11 mm.) C, Single-subject parameter estimates for the difference between responses to behaviorally relevant and irrelevant value, averaged across the whole of the right (top) and left (bottom) anatomically defined ventral striatum ROIs (positive values reflect greater activation for relevant than irrelevant value). Significantly greater activation was observed to relevant than irrelevant value.

Figure 2.

A, Time course of a single trial. After a jittered fixation interval (1000–1500 ms), subjects were presented with text indicating which properties of stimuli determined the reward contingencies on that trial (de facto which “condition” they were in). Thus, the text said “Visual” for trials where the visual cue determined the outcome of the trial, “Auditory” for trials where the auditory cue determined the outcome, and either “Congruent” or “Incongruent” where the combination of both determined the outcome. Each subject saw only one of Congruent or Incongruent, and these were counterbalanced across subjects to decorrelate value and congruence. Following a brief (2000 ms) interval, one of two patterned boxes and one of two synthesizer pads were presented to the subject for 2000 ms, who then chose to accept or reject the offer before its offset. Outcomes were presented visually for 1200 ms after a variable delay (between 2000 and 8000 ms). The subject's current winnings during the session were indicated by the length of a bar constantly displayed at the bottom of the screen. B, Illustrates possible outcomes. Subjects were shown the outcome of the trial (indicated by text saying either “WIN” or “LOSE”) and whether they chose to accept or reject an offer made to them. If they chose to accept the offer, the bar at the bottom of the screen indicated whether their cumulative earnings during the session increased or decreased in length by equal amounts according to whether they won or lost. If they chose to reject the offer in contrast, the text was presented with a line through it to indicate that this choice did not affect their earnings and the cumulative earnings bar did not change in length. C, Behavioral results. Logistic regression analysis demonstrated that in all three task conditions, behaviorally relevant stimulus properties (dark gray) had a stronger effect on behavior than irrelevant ones (light gray). Error bars indicate bootstrapped 90% confidence intervals.

Figure 3.

Results of the searchlight decoding analysis for visual stimuli in task-relevant and -irrelevant conditions. Significant decoding was possible in visual cortex for both relevance conditions, and conjunction analysis revealed a large area of overlap in visual cortex. This suggests that visual stimuli (whether task relevant or not) are represented in a similar fashion in sensory areas. (Images thresholded at p < 0.001 uncorrected; blue irrelevant, green relevant.)

Figure 4.

A, Outcome activity in ventral striatum was correlated positively with obtained wins (top) and negatively with foregone wins (bottom), consistent with a role in signaling whether a subject had performed the correct action or not. (Image thresholded at p < 0.005 uncorrected for display purposes; Y = 11 mm.) B, Activity in the dorsomedial prefrontal cortex and bilateral anterior insula correlated negatively with obtained wins (top) and positively with foregone ones (bottom), consistent with a role in error signaling. (Image thresholded at p < 0.005 uncorrected for display purposes; Y = 23 mm, X = 0 mm.) C, Illustration of outcome responses. Mean parameter estimates extracted from voxels in the left and right ventral striatum showing the strongest positive responses to obtained outcomes (top), and from voxels in the left and right anterior insula showing the strongest negative responses to obtained outcomes (bottom). Activity in as the ventral striatum and anterior insula showed opposite patterns of responding to the wins minus losses contrast in the obtained (accept, green) and forgone (reject, red) conditions. (Note that these plots are illustrative only, and all inference was performed using the standard SPM analysis.) Error bars indicate 95% confidence intervals.