Interaction between orbital prefrontal and rhinal cortex is required for normal estimates of expected value

Abstract

Predicting and valuing potential rewards requires integrating sensory, associative, and contextual information with subjective reward preferences. Previous work has identified regions in the prefrontal cortex and medial temporal lobe believed to be important for each of these functions. For example, activity in the orbital prefrontal cortex (PFo) encodes the specific sensory properties of and preferences for rewards, while activity in the rhinal cortex (Rh) encodes stimulus-stimulus and stimulus-reward associations. Lesions of either structure impair the ability to use visual cues or the history of previous reinforcement to value expected rewards. These areas are linked via reciprocal connections, suggesting it might be their interaction that is critical for estimating expected value. To test this hypothesis, we interrupted direct, intra-hemispheric PFo-Rh interaction in monkeys by performing crossed unilateral ablations of these regions (functional disconnection). We asked whether this circuit is crucial primarily for cue-reward association or for estimating expected value per se, by testing these monkeys, as well as intact controls, on tasks in which expected value was either visually cued or had to be inferred from block-wise changes in reward size in uncued trials. Functional disconnection significantly affected performance in both tasks. Specifically, monkeys with functional disconnection showed less of a difference in error rates and reaction times across reward sizes, in some cases behaving as if they expected rewards to be of equal magnitude. These results support a model whereby information about rewards signaled in PFo is combined with associative and contextual information signaled within Rh to estimate expected value.

Figure 1.

Schematic illustration of the three instrumental tasks. In the visually cued task (top row) each reward size was paired with a unique cue image presented at the start of each trial. To earn the reward, monkeys had to release a lever a short time after the color of a target stimulus changed from red to green. In the visually instructed task (middle row), monkeys were required to react to the color change in the target stimulus but were not given a visual cue to reward size. In both the visually cued and visually instructed tasks, correct performance was followed by visual feedback and reward delivery, error trials were repeated until performed correctly. In the self-initiated task (bottom row), monkeys simply had to press and release a lever at their own pace to earn reward. They were given visual feedback after a rewarded bar release (bar releases occurring during the reward period were not reinforced). In both the visually instructed and self-initiated tasks, reward size varied across blocks of trials (25 trials per block).

Figure 2.

Lesion reconstructions. Intended PFo and Rh lesions are shown on a ventral view of the macaque brain as well as on coronal sections at the indicated levels. Estimates of the extent of aspiration lesions for two of the four monkeys in the PFo X Rh group are plotted on coronal sections at the indicated levels and reconstructed onto a ventral view of the macaque brain; reconstructions for each case are shown at the top of each column. Lesions were reconstructed using MR images; representative MR images for monkey P are shown next to the corresponding coronal section for both the unilateral PFo and Rh lesion. Yellow arrows mark the lesion boundaries. The two monkeys not shown received the opposite pattern of PFo and Rh lesions (PFo left hemisphere, Rh right hemisphere). Across the group, lesions largely covered the areas of interest, and damage to adjacent structures was minimal and distributed idiosyncratically across monkeys.

Figure 3.

Preoperative performance in the visually cued task. A, Group average performance in the visually cued task is plotted as error rates (ordinate) versus reward size (abscissa), data are from the final 15 sessions of testing before the first unilateral lesion. There was a significant effect of reward size on performance, with error rates decreasing with reward size. B, Overall performance (collapsing across reward sizes) for the same 15 sessions shown in A is plotted as error rates (ordinate) versus normalized accumulated reward (abscissa, see Methods and Materials, Data analysis). There was a significant effect of accumulated reward on performance, with error rates increasing as monkeys became more sated. C, Performance for the same 15 sessions shown in A and B is plotted as error rates (ordinate) versus normalized accumulated reward (abscissa) separately for each reward size. There was a significant interaction between the effects of reward size and accumulated reward. D, Heat maps depicting group average error rates (color scale) as a function of reward size (ordinate) and session number (abscissa). Data in each panel are from a different cue set. There was a significant interaction between the effects of reward size, session number and cue set. On average, performance stabilized earlier in testing for later cue sets. E, Heat maps depicting group average error rates (color scale) as a function of accumulated reward (each session divided into sextiles, ordinate) and session number (abscissa). Data in each panel are from a different cue set (4 preoperative cue sets). The effect of accumulated reward was not significantly different across sessions or cue sets. Norm. Accum. Reward, Normalized accumulated reward.

Figure 4.

Estimating learning curves in the visually cued task. A, In each panel, group average error rates (ordinate) are plotted against reward size (abscissa), solid curves are the best fitting hyperbolic model. Within a row, data is from separate sessions with the same cue set, within a column, data is from the same session number but different cue set (cue set and session number are indicated in each panel). During training, the rate at which performance changed with additional testing on a given cue set varied across cue sets. B, Learning curves are plotted as the goodness-of-fit (R²) of the best-fit hyperbolic model (ordinate) versus session number separately for each cue set (smoothed with a 3 session moving average filter). Animals took significantly longer to achieve asymptotic performance on the first cue set. hyp, Hyperbolic.

Figure 5.

PFo-Rh functional disconnection disrupts performance in the visually cued task. A, In each panel, group average error rates (ordinate) are plotted against normalized accumulated reward (abscissa), data are from the final preoperative and unilateral ablation conditions (middle panel, old cue set; right panel, new cue set). There was no significant effect of unilateral PFo or Rh ablations. B, Conventions as in A but for data collected following PFo-Rh functional disconnection. Unlike unilateral ablation of PFo or Rh, PFo-Rh functional disconnection had a significant effect on performance. C, Preoperative and postoperative data from testing with the old cue set are plotted as error rates (ordinate) versus normalized accumulated reward (abscissa) separately for each reward size. Following PFo-Rh functional disconnection, monkeys made fewer errors in 1 drop trial early in a session, and more errors in 2, 4, and 8 drop trials late in a session. D, Goodness-of-fit for the best fit hyperbolic model (ordinate) is plotted versus session number (abscissa) for data from the final preoperative training set and the unilateral (middle) and PFo X Rh (right) conditions. Norm. Accum. Reward, Normalized accumulated reward.

Figure 6.

Doubling the number of cue–reward associations results in a significant impairment in the visually cued task. A, Performance in the visually cued task is plotted as error rate (ordinate) against reward size (abscissa), data are from conditions with either 4 unique cues and 4 reward sizes (top) or 8 unique cues and 4 unique reward sizes (bottom). Data in the left (right) panel are from the control (PFo X Rh) group. The PFo X Rh group made significantly fewer errors—and exhibited less of a difference in performance across reward sizes—in the 8 cue condition. B, Goodness-of-fit for the best fit hyperbolic model (ordinate) is plotted versus session number (abscissa) for both the control (left) and PFo X Rh (right) groups. C, Error rates for the two cues that signaled a given reward size are plotted versus one another, separate panels are data from the control (left) and PFo X Rh (right) groups. Norm. Accum. Reward, Normalized accumulated reward.

Figure 7.

PFo-Rh functional disconnection disrupts performance in the self-initiated task. A, Self-initiated performance is plotted as release interval (see Materials and Methods, Data analysis) (ordinate) versus reward size (abscissa) for the control and PFo X Rh group. The control group, unlike the PFo X Rh group, exhibited a significant decrease in release interval with increasing reward size. B, Within-block performance is plotted as standardized release interval (see Materials and Methods; ordinate) versus within-block trial number (abscissa) for the PFo X Rh (left) and control (right) groups. By definition, responses faster than average are positive, responses slower than average are negative. For the control, but not PFo X Rh, group responses separate according to reward size immediately (second trial of a block) after block transitions. C, Average release intervals (ordinate) are plotted versus normalized accumulated reward (abscissa) separately according to reward size. Left, PFo X Rh group; right, control group. Norm. Accum. Reward, Normalized accumulated reward; RI, release interval; msec, milliseconds.

Figure 8.

Introducing a visual reward cue does not rescue performance in the self-initiated task. A, Cued self-initiated performance is plotted as release interval (ordinate) versus reward size (abscissa) for the control and PFo X Rh group. As for the standard self-initiated task, the control but not PFo X Rh, group exhibited a significant decrease in release interval with increasing reward size. B, The trial-by-trial dynamics of the responses of the PFo X Rh (left panel) and control (right panel) groups in the cued self-initiated task are plotted as standardized release intervals (ordinate) versus within-block trial number (abscissa). C, Average release intervals (ordinate) for the trials highlighted within the rectangular regions in (B) are plotted versus reward size (abscissa). For the PFo X Rh group, release intervals on first trials in a block are significantly faster than release intervals on subsequent trials. Additionally, for the PFo X Rh but not control group, release intervals tend to decrease with reward size only on first trials. Unlike the PFo X Rh group, control group responses were not faster on first trials in a block and actually showed a more straightforward relationship with reward on later trials (compare responses on fourth with first trials). RI, Release interval; msec, milliseconds.

Figure 9.

An imperative cue partially rescues performance. A, Performance in the visually instructed task is plotted as both error rates and reaction times (left, right; ordinate) versus reward size (abscissa) for the PFo X Rh and control groups. Reaction times were significantly affected by reward size for both groups. B, Trial-by-trial response dynamics are plotted as standardized reaction times (ordinate) versus within-block trial number (abscissa) for the PFo X Rh (left) and control (right) groups. C, Group average error rates (top row) and reaction times (bottom rows; ordinate) are plotted versus normalized accumulated reward (abscissa) separately according to reward size. Left, PFo X Rh group; right, control group. RT, reaction time; msec, milliseconds; Norm. Accum. Reward, Normalized accumulated reward.

Figure 10.

Effect of bilateral PFo and Rh lesions. A, Performance in the visually cued task is plotted separately—as error rates (ordinate) versus normalized accumulated reward (abscissa)—for the control (left), bilateral PFo (middle), and bilateral Rh (right) groups. Both the bilateral PFo and bilateral Rh groups were significantly different from controls in this task. B, Performance in the self-initiated task is plotted as release interval (ordinate) versus reward size (abscissa), separate traces correspond to data from the control and bilateral lesion groups. Bilateral PFo and bilateral Rh lesions significantly altered monkeys' performance in the self-initiated task. C, Trial-by-trial response dynamics are plotted as standardized release interval (ordinate) versus within-block trial number (abscissa) for the bilateral PFo (middle) and bilateral Rh (right) groups (see Fig. 7B for comparison with control group response dynamics). D, Group average release intervals (ordinate) are plotted versus normalized accumulated reward (abscissa) separately according to reward size. Left, PFo group; right, Rh group. RI, release interval; msec, milliseconds; Norm. Accum. Reward, normalized accumulated reward.