Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes

Abstract

Advantageous foraging choices benefit from an estimation of two aspects of a resource's value: its current desirability and availability. Both orbitofrontal and ventrolateral prefrontal areas contribute to updating these valuations, but their precise roles remain unclear. To explore their specializations, we trained macaque monkeys on two tasks: one required updating representations of a predicted outcome's desirability, as adjusted by selective satiation, and the other required updating representations of an outcome's availability, as indexed by its probability. We evaluated performance on both tasks in three groups of monkeys: unoperated controls and those with selective, fiber-sparing lesions of either the OFC or VLPFC. Representations that depend on the VLPFC but not the OFC play a necessary role in choices based on outcome availability; in contrast, representations that depend on the OFC but not the VLPFC play a necessary role in choices based on outcome desirability.

Keywords: decision-making; learning; macaque; orbitofrontal; prefrontal cortex; reward; ventrolateral.

Figure 1. The three-choice probabilistic learning task, reward schedules, and lesions extents

A) Task sequence. On each trial, monkeys were presented with three stimuli for choice, and through trial and error could learn which stimulus was associated with the highest probability of reward. B) Reward delivery was dependent on the underlying reward schedules shown here and the ones illustrated in Fig. S3. C) Schematic of OFC (green) and VLPFC lesions (blue). For both OFC and VLPFC lesions, T2-weighted MRI images taken within one week of surgery was used to estimate the extent of the lesions. White hypersignal in the T2-weighted images—set off by arrowheads—is associated with edema that follows injections of excitotoxins and indicates the likely extent of the lesion. For T2-weighted images, left and right sides of the MR images are from different scans and have been placed together for ease in viewing. Yellow dashed lines indicate where images from two different postoperative scans have been joined. MR images from T1-weighted scans acquired at least a year after surgery confirm the loss of cortex in the intended regions. Numerals indicate the distance in mm from the interaural plane. MRI images are from levels matching the drawings of coronal sections.

Figure 2. VLPFC, but not OFC, lesions disrupt the ability to choose according to outcome probability on the three-choice probabilistic learning task

A) Mean (±SEM) choice behavior of unoperated controls (gray, top row, n = 8), monkeys with OFC lesions (green, top row, n = 4), and monkeys before (gray, n =4) and after (blue, n= 4) VLPFC lesions (bottom row) on schedule 2. Note that in A (top), the gray curve and shading (Control) is largely obscured by the overlying green curve and shading (OFC). Colored points represent the identity and probability of receiving a reward for selection of the high reward option. B) Mean (±SEM) probability of choice of reinforcement learning estimated high reward option in the first and second sets of 150 trials for unoperated controls (n = 8) and monkeys with OFC lesions (top row, n= 4) and monkeys before and after VLPFC lesions (bottom row, n = 4) on schedule 2. Symbols show scores of individuals subjects. C) Mean (±SEM) probability of switched choice options from trial-to-trial for unoperated controls (n = 8) and monkeys with OFC lesions (top row, n = 4) and monkeys before and after VLPFC lesions (bottom row, n = 4) on schedule 2. * p<0.05. Also see Figs S2 and S3.

Figure 3. VLPFC, but not OFC, lesions disrupt probabilistic learning

A) Mean (±SEM) probability of choice of the option associated with the highest probability of reward as defined by a reinforcement learning model fit to monkeys’ choices in each of the 4 schedules for unoperated controls (gray, top row, n = 8) and monkeys with OFC lesions (green, top row, n = 4) and monkeys before and after VLPFC lesions (gray and blue, respectively, bottom row, n = 4). B) Mean (±SEM) probability of switching on rewarded (darker shading) or unrewarded trials (lighter shading) for unoperated controls (gray, n = 8), monkeys with OFC lesions (green, n = 4), and monkeys before (gray, n = 4) and after VLFPC lesions (blue, n = 4). Symbols show scores of individual subjects.

Figure 4. VLPFC, but not OFC lesions, disrupt contingent and noncontingent learning

A) Schematic of the full matrix of five previous choices and corresponding rewards received for those choices. The matrix components highlighted in red along the diagonal represent the influence of previous choices and their contingent rewards on subsequent choices. Components highlighted in green represent the influence of rewards from the previous five trials and the most recent choice on subsequent choices whereas those highlighted in blue represent the influence of the five previous choices and the reward from the previous trial on the subsequent choices. B) Matrix plots showing the influence (beta weightings from logistic regression) of all combinations of the five previous choices and rewards on subsequent choice for control monkeys (n = 8), monkeys with OFC lesions (top row, n = 4), and monkeys before and after VLPFC (bottom row, n = 4). Lighter shading is associated with higher beta weights. C–E) Raw beta weights from the matrix for controls (gray, n = 8) and monkeys with OFC lesions (green, top row, n = 4) and monkeys before (gray) and after VLPFC lesions (blue, bottom row, n = 4). * p<0.05.

Figure 5. Direct comparison of contingent learning in monkeys with OFC and VLPFC lesions

Mean (±SEM) contingent learning difference score for monkeys with OFC (green, n = 4) and VLPFC lesions (blue, n = 4). For each subject, we computed difference scores based on the beta weights from the logistic regression that reflect contingent associations (red cells in Fig 4A) as follows: for monkeys with excitotoxic OFC lesions, the control group mean was subtracted from each OFC lesion monkey’s individual score, whereas for monkeys that received VLPFC lesions, difference scores were computed as the difference between each subject’s preoperative and postoperative test scores. Negative scores reflect decrease in performance relative to controls/preoperative data. * p<0.05.

Figure 6. OFC lesions, but not VLPFC lesions, disrupt the ability to choose according to outcome value on the reinforcer devaluation task

A) Mean (±SEM) number of errors for each group during the first 10 sessions of the 60 pair discrimination learning. Inset shows the total errors to criterion for unoperated controls (n = 8), monkeys with OFC lesions (n = 4), and monkeys with VLPFC lesions (n = 4). B–C) Mean (±SEM) proportion shifted for unoperated controls (gray bars, n = 8), monkeys with OFC lesions (green bars, n = 4), and monkeys with VLPFC lesions (blue bars, n = 4) during (B) the two reinforcer devaluation tests (C) and control test where only foods (no objects) were presented for choice. In all plots symbols show scores of individual subjects. * p<0.05.