Does the orbitofrontal cortex signal value?

Abstract

The orbitofrontal cortex (OFC) has long been implicated in associative learning. Early work by Mishkin and Rolls showed that the OFC was critical for rapid changes in learned behavior, a role that was reflected in the encoding of associative information by orbitofrontal neurons. Over the years, new data-particularly neurophysiological data-have increasingly emphasized the OFC in signaling actual value. These signals have been reported to vary according to internal preferences and judgments and to even be completely independent of the sensory qualities of predictive cues, the actual rewards, and the responses required to obtain them. At the same time, increasingly sophisticated behavioral studies have shown that the OFC is often unnecessary for simple value-based behavior and instead seems critical when information about specific outcomes must be used to guide behavior and learning. Here, we review these data and suggest a theory that potentially reconciles these two ideas, value versus specific outcomes, and bodies of work on the OFC.

Figure 1

OFC contribution to Pavlovian overexpectation, adapted from Takahashi et al. Shown is the experimental timeline linking conditioning, compound conditioning, and probe phases to data from each phase. Top and bottom rows of plots indicate control (saline infusion into the OFC) and OFCi group (muscimol + baclofen infusion into the OFC), respectively. In timeline and plots, V1 is a visual cue (a cue light); A1, A2, and A3 are auditory cues (tone, white noise, and clicker, counterbalanced), and O1 and O2 are different flavored sucrose pellets (banana and grape, counterbalanced). Position of the cannula within OFC in saline controls (gray dot) and OFCi (black dot) rats are shown beneath the timeline. (A) Percentage of individuals responding to food cup during cue presentation across 10 days of conditioning. Gray, black, and white squares indicate A1, A2, and A3 cues, respectively. (B) Percentage of those responding to food cup during cue presentation across four days of compound training. Gray, black, and white squares indicate A1/V1, A2, and A3 cues, respectively. Gray and black bars in the insets indicate average normalized percentage responding to A1/V1 and A2, respectively. (C) Percentage of those responding to food cup during cue presentation in the probe test. Line graph shows those responding across the eight trials, and the bar graph shows average number responding in these eight trials. Gray, black, and white colors indicate A1, A2, and A3 cues, respectively. * P < 0.05 and ** P < 0.01 on post hoc contrast testing. NS, not significant. Error bars denote SEM.

Figure 2

OFC contribution to reinforcer-specific Pavlovian to instrumental transfer adapted from Ostlund et al. Pavlovian training consisting of pairing two different auditory cues with either sucrose or pellet rewards. Sham or OFC lesions were given before or following Pavlovian training. Rats were trained that a right lever press produced pellets while a left lever press produced sucrose solution (counterbalanced). Each rat was tested twice, each with one lever present. Mean lever-press rates are shown during a cue-free baseline period (white bars), a cue signaling the same outcome as the lever press (black bar), and a cue signaling a different outcome (gray bar). Pre means Sham or OFC lesion was given before Pavlovian conditioning; post means Sham or OFC lesion was given following Pavlovian conditioning. Error bars denote SEM.

Figure 3

OFC contribution to conditioned reinforcement adapted from Burke et al. Shown is the experimental timeline linked to data from each conditioned reinforcement (CRf) test. In the timeline and figures, A, B, X, and Y are training cues; R1 and R2 are instrumental responses; and O1 and O2 are different flavored sucrose pellet reinforcers. Normal (open bars) and OFC-lesioned rats (black bars) were first trained to associate two different auditory cues (A and B) with two differently flavored sugar pellets (banana and grape). After these associations were learned, an “unblocking” phase occurred in which A was compounded with a novel cue, and X and B compounded with a novel cue Y. While AX predicted the original flavor, BY predicted another flavor. This arrangement has been shown to “block” learning to X but “unblock” learning to Y. (A–D) Lever pressing for A versus X, or Y versus X, in control (open bars) and lesioned (filled bars) rats before (A, B) and after (C, D) devaluation. Lesions diminished responding for Y before devaluation (A, B); controls diminished lever pressing for Y after devaluation (C, D). Lever pressing is averaged across two 30-min sessions in each figure. Asterisks indicate significance at P < 0.05 on post hoc contrast testing; the gray numbers indicate the ratio of responding on the two levers for each significant comparison. NS, not significant. Error bars denote SD.

Figure 4

OFC contribution to differential outcome expectancy; adapted from McDannald et al. (A) Sham and OFC-lesioned rats were required to learn the following stimulus-action-outcome associations: S1 → R1 → O1, S2 → R2 → O2, and S3 → R3 → O1/O2. S1–3 were different auditory cues, R1–3 different operant responses, and O1,2 different flavors of sucrose. Performance during S1, in which R1 responses were always reinforced with O1, and errors, consisting of R2 or R3 responses, were not reinforced. R2 responses were always reinforced with a different outcome (O2) on S2 trials, whereas R3 was reinforced with the same outcome (O1) as R1 on one-half of the S3 presentations and with O2 on the other half of the S3 trials. (B) Responding during S1 trials is shown. Sham rats showed more shared-outcome errors (R3) than different-outcome errors (R2) during S1. This result can be attributed to the rats' use of specific outcome expectancies to guide responding: responses reinforced in the presence of different expectancies (e.g., R1 and R2) were differentiated more readily than responses that yielded the same outcome (e.g., R1 and R3). In contrast, OFC-lesioned rats showed equivalent levels of shared- and different-outcome errors. The absence of any difference in shared- and different-outcome errors in these rats is consistent with impairment in the associative basis for outcome expectancy learning. Error bars denote SEM.

Figure 5

OFC contribution to value and identity unblocking; adapted from McDannald et al. (A) Rats were first trained to associate three different visual cues (A, B, C) with three different amounts (3 or 1) and identities (O1 or O2; banana-flavored sugar pellets or grape-flavored sugar pellets) of reward: A → 3xO1, B → 3xO2, and C → O1. Following this phase an unblocking phase was given in which compounds of the original cues and three novel auditory cues, AX, BY and CZ, all predicted 3xO1. In this way X signaled the expected reward (Blocked), Y signaled a differently flavored but similarly valued reward (Identity) and Z signaled a differently valued but similarly flavored reward (Value). (B) Food cup responding to the Blocked cue (which signaled the expected number of pellets) and the Value cue (which signaled greater than expected reinforcement) for Control and OFC-lesioned rats. (C) Food cup responding to the Blocked cue (which signaled the expected flavor of pellets) and the Identity cue (which signaled an equally preferred but differently flavored pellet) for Control and OFC-lesioned rats. (D) The relationship between value sensitivity to cues A and B (which predicted equal amounts of different-flavored sugar pellets) in initial conditioning and the display of identity unblocking for Control and OFC-lesioned rats. Asterisks indicate significance at P < 0.05 on post hoc contrast testing. NS, not significant. Error bars denote SEM.