Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex

Abstract

Knowledge about hypothetical outcomes from unchosen actions is beneficial only when such outcomes can be correctly attributed to specific actions. Here we show that during a simulated rock-paper-scissors game, rhesus monkeys can adjust their choice behaviors according to both actual and hypothetical outcomes from their chosen and unchosen actions, respectively. In addition, neurons in both dorsolateral prefrontal cortex and orbitofrontal cortex encoded the signals related to actual and hypothetical outcomes immediately after they were revealed to the animal. Moreover, compared to the neurons in the orbitofrontal cortex, those in the dorsolateral prefrontal cortex were more likely to change their activity according to the hypothetical outcomes from specific actions. Conjunctive and parallel coding of multiple actions and their outcomes in the prefrontal cortex might enhance the efficiency of reinforcement learning and also contribute to their context-dependent memory.

Figure 1. Behavioral task and payoffs

A. Temporal sequence of a rock-paper-scissors task and example feedback displays. The positions of targets corresponding to rock (R), paper (P), scissors (S) are indicated in the frame for the delay period for an illustrative purpose (see also Figure S1). B. Payoff matrix in which each number refers to the amount of juice (×0.2 ml) delivered to the animal. C. Feedback colors used to indicate the payoffs.

Figure 2. Learning from actual and hypothetical payoffs

A. Effects of actual payoffs on choices. All 3 animals were more likely to choose the same target again after winning (abscissa) than losing or tying (ordinate). B. Effects of hypothetical payoffs on choices. After both tie and loss trials, all 3 animals were more likely to choose the target that would have been a winning choice than the other unchosen target (tying and losing target in loss and tie trials, respectively). See also Figure S2 and Table S1.

Figure 3. Example neurons with activity related to hypothetical outcomes

A. Spike density functions (SDF, convolved with a Gaussian filter with σ=40ms) of a DLPFC neurons estimated separately according to the position (columns) and payoff (line colors) of the winning target and the position of the target chosen by the animal (rows). Position of the winning target (W) and the payoffs from the other targets are indicated in the top panels, and the mean firing rates during the feedback period (gray shaded regions in the SDF plots) are shown at the bottom. Shaded areas and error bars, SEM. B. Activity of an OFC neuron in the same format as in A. For the neuron in A, the positions of losing and tying targets were fixed for each winning target (Experiment I), whereas for the neuron in B, the positions of losing and tying targets were counter balanced (Experiment II). See also Figure S3.

Figure 4. Anatomical locations of neurons with outcome effect

A. Locations of DLPFC neurons that showed significant changes in their activity related to actual and hypothetical outcome irrespective of whether they were linked to specific choices or not. B. Locations of OFC neurons. The positions of the recorded neurons were estimated and plotted on a horizontal plane. The neurons shown medially from MOS were not in the ventral surface but in the fundus of MOS. MOS, medial orbital sulcus. LOS, lateral orbital sulcus. TOS, transverse orbital sulcus. The number of neurons recorded from each area is shown. See also Figure S4.

Figure 5. Effect size for the activity related to actual and hypothetical outcomes

A. CPD related to actual and hypothetical outcomes are shown separately according to whether their effects vary across different actions (AOC/HOC) or not (AON/HON). B. Difference between choice-specific and choice-unspecific CPD for actual (AOC–AON) and hypothetical (HOC–HON) outcomes. The legend shows the number of neurons included in this analysis. *, p<0.01 (two-tailed t-test). See also Table S2.

Figure 6. Time course of outcome-related activity

Time courses of CPD for actual and hypothetical outcomes plotted separately according to whether their effects on neural activity changed significantly for different actions (B) or not (A). Numbers in the parentheses refer to the number of neurons with significant effects in each cortical area. Shaded areas, SEM. See also Table S4.