Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation

Abstract

We monitored single-neuron activity in the orbitofrontal cortex of rats performing a time-discounting task in which the spatial location of the reward predicted whether the delay preceding reward delivery would be short or long. We found that rewards delivered after a short delay elicited a stronger neuronal response than those delivered after a long delay in most neurons. Activity in these neurons was not influenced by reward size when delays were held constant. This was also true for a minority of neurons that exhibited sustained increases in firing in anticipation of delayed reward. Thus, encoding of time-discounted rewards in orbitofrontal cortex is independent of the encoding of absolute reward value. These results are contrary to the proposal that orbitofrontal neurons signal the value of delayed rewards in a common currency and instead suggest alternative proposals for the role this region plays in guiding responses for delayed versus immediate rewards.

Figure 1. Experimental Design

Choice task during which we varied (A) the delay preceding reward delivery and (B) the size of the reward. Figure shows the sequence of events in each trial in four blocks in which we manipulated the time to reward or the size of the reward. See Experimental Procedures for a description of the task.

Figure 2. Impact of Delay Length and Reward Size on Behavior

(A) Average choice rate, collapsed across direction, for all sessions for trials before and after the switch from short to long delay. (Inset) The height of each bar indicates the percent choice of short delay and long delay taken over all choice trials. (B and C) The height of each bar indicates the percent correct (B) and reaction time (C) across all recording sessions in all rats on short delay (gray) and long delay (black) forced choice trials. (D–F) Impact of reward size on the same behavior measures described in (A)–(C). *p < 0.05, t test. Error bars = SE.

Figure 3. Impact of Delay Length and Reward Size on All 159 Reward-Related Neurons

(A) Location of recording sites in rat OFC. Vertical bars on the drawing indicate the center of the electrode track in each rat; boxes indicate approximate extent of recording sessions vertically during transition through each area and give an estimate of the spread of the wires (~1 mm). (B and C) Curves representing mean population firing rate during performance of forced choice trials for all 159 reward-responsive neurons as a function of time under conditions where delay length varied (short = blue, long = red). (Data is aligned on reward delivery in [B] and well entry in [C]). (D) Curves representing mean population firing rate during performance of forced choice trials for the same 159 reward-responsive neurons as a function of time under conditions where magnitude of the reward varied (big = green, small = orange). Data is aligned on reward delivery. Solid and dashed lines indicate the neuron’s preferred and nonpreferred direction. For each neuron, the direction that yielded the stronger response (averaged over all trials) was designated as preferred. (Insets) Impact of delay length ([B] and [C]) and reward size (D) on 104 reward-responsive neurons during performance of free choice trials only. Data in insets include sessions during which rats performed at least two free choice trials per condition.

Figure 4. Incidence of Significant Effects of Delay Length and Reward Size

Incidence of significant effects of delay length (top panels) and reward size (bottom panels) for responses made to the left ([A]–[D]) and right well ([E]–[H]). This analysis was performed independently for each direction. For counts of neurons that exhibited selectivity in both directions, see Table 1. (Top panels) Pie charts represent counts of neurons exhibiting significant effects in a t test (p < 0.05), taking firing rate during the “reward epoch” as the dependent variable and employing delay length (short or long) as a factor. Bar graphs represent counts of neurons exhibiting significant effects of delay (t test; p < 0.05) during the response epoch. Bar graphs are split into two categories, based on their selectivity during the reward epoch: ([B] and [F]) those that fired more for rewards delivered after a short delay and ([C] and [G]) those that fired more for rewards delivered after a long delay. The reward epoch started 250 ms prior to reward and ended 1000 ms after reward. The response epoch started from 250 to 1500 ms after a response at the fluid well. (Bottom panels) Counts of neurons exhibiting significant effects in a t test (p < 0.05), taking firing rate during the reward epoch as the dependent variable and employing reward size (big and small) as a factor. Blue: short > long. Red: long > short. Green: big > small. Orange: small > big. * p < 0.05; chi-square.

Figure 5. Population of OFC Neurons that Fired More Strongly after Short Delays

(A) Population histogram representing firing rate as a function of time during the trial for neurons that fired more strongly during the reward epoch after short delays (n = 65). Activity is aligned on well entry. We plot the 3 s delay as an example of delayed reward because it was included in all recording sessions. Blue: short. Red: long. Solid: preferred direction. Dashed: nonpreferred direction. (B) Single-cell example of a short delay-preferring neuron. Activity is plotted for the last ten trials in a block in which reward was delivered in the cell’s preferred direction after 500 ms (blue), followed by trials in which the reward was delayed by 1–4 s (red). Each row represents a single trial, each tick mark represents a single action potential, and the colored lines indicate when reward was delivered. (C) Population histogram of same 65 neurons (shown in [A]) during trials when reward size varied. Green: big. Orange: small. Solid: preferred direction. Dashed: nonpreferred direction. (D) Relation of firing dependent on delay length to firing dependent on reward size for those neurons that fired more strongly after short delays. The delay index and reward index are computed on the basis of firing during the reward epoch. Delay index = (S − L)/(S + L), where S and L represent firing rates on short and long delay trials, respectively. Reward index = (B − S)/(B + S), where B and S represent firing rates on big and small reward trials, respectively.

Figure 6. Population of OFC Neurons that Fired More Strongly after Long Delays

(A) Population histogram representing firing rate as a function of time during the trial for neurons that fired more strongly during the reward epoch after long delays (n = 27). (B) Single-cell example. (C) Population histogram of same 27 neurons (shown in [A]) during trials when delay was held constant but reward size varied. (D) Relation of firing dependent on delay length to firing dependent on reward size for those neurons that fired more strongly after long delays. Average baseline firing rate and distribution of baseline firing rates for neurons recorded in this group (3.130 spikes/s) were not significantly different from those reported in Figure 5 (2.864 spikes/s; Wilcoxon, p = 0.4062). Conventions are the same as in Figure 5.

Figure 7. Dependency of Firing Rate during the Reward Epoch on Delay Length and Future Choice Probability as Revealed by Multiple Regression Analysis

(A) Blue and red represent cases in which the correlation between firing rate and delay length was negative (stronger firing for short delay) or positive (stronger firing for long delay), respectively. (B and C) Each bar represents the number of neurons in which the correlation between firing rate and future choice probability was positive (more likely to choose direction associated with short delay) or negative (less likely to choose direction associated with short delay) for those cells that also showed (B) a negative or (C) a positive correlation with delay length. * p < 0.05; chi-square.