Neurobiology of economic choice: a good-based model

Abstract

Traditionally the object of economic theory and experimental psychology, economic choice recently became a lively research focus in systems neuroscience. Here I summarize the emerging results and propose a unifying model of how economic choice might function at the neural level. Economic choice entails comparing options that vary on multiple dimensions. Hence, while choosing, individuals integrate different determinants into a subjective value; decisions are then made by comparing values. According to the good-based model, the values of different goods are computed independently of one another, which implies transitivity. Values are not learned as such, but rather computed at the time of choice. Most importantly, values are compared within the space of goods, independent of the sensorimotor contingencies of choice. Evidence from neurophysiology, imaging, and lesion studies indicates that abstract representations of value exist in the orbitofrontal and ventromedial prefrontal cortices. The computation and comparison of values may thus take place within these regions.

Figure 1

Good-based model. The value of each good is computed integrating multiple determinants, of which some are external (e.g., commodity, quantity, etc.) and other are internal (motivation, im/patience, etc.). Offer values of different goods are computed independently of one another and then compared to make a decision. This comparison takes place within the space of goods. The choice outcome (chosen good, chosen value) then guides an action plan through a good-to-action transformation. Values and choice outcomes also inform other brain systems, including sensory systems (through perceptual attention), learning (e.g., through mechanisms of reinforcement learning) and emotion (including autonomic functions).

Figure 2

Measuring subjective values: value encoding in the OFC. a. Economic choice task. In this experiment, monkey chose between different juices offered in variable amounts. Different colors indicated different juice types and the number of squares indicated different amounts. In the trial depicted here, the animal was offered 4 drops of peppermint tea (juice B) versus 1 drop of grape juice (juice A). The monkey indicated its choice with an eye movement. b. Choice pattern. The x-axis represents different offer types ranked by the ratio #B:#A. The y-axis represents the percent of trials in which the animal chose juice B. The monkey was roughly indifferent between 1A and 4B. A sigmoid fit indicated, more precisely, that 1A = 4.1B. The relative value (4.1 here) is a subjective measure in multiple senses. First, it depends on the two juices. Second, for given two juices, it varies for different individuals. Third, for any individual and two given juices, it varies depending, for example, on the motivational state of the animal (thirst). Thus to examine the neural encoding of economic value, it is necessary to examine neural activity in relation to the subjective values measured concurrently. c. OFC neuron encoding the offer value. Black circles indicate the behavioral choice pattern (relative value in the upper left) and red symbols indicate the neuronal firing rate. Red diamonds and circles refer, respectively, to trials in which the animal chose juice A and juice B. There is a linear relationship between the activity of the cell and the quantity of juice B offered to the monkey. d. OFC neuron encoding the chosen value. There is a linear relationship between the activity of the cell and the value chosen by the monkey in each trial. For this session, 1A=2.4B. The activity of the cell is low when the monkey chooses 1A or 2B, higher when the monkey chooses 2A or 4B, and highest when the monkey chooses 1A or 6B. Neurons encoding the chosen value are thus indentified based on the relative value of the two juices. e. OFC neuron encoding the taste. The activity of the cell is binary depending on the chosen juice but independent of its quantity. (2d–e, same conventions as in 2c.) Adapted from Padoa-Schioppa and Assad (2006) Nature (Nature Publishing Group) and from Padoa-Schioppa (2009) J Neurosci (Soc for Neurosci, with permission).

Figure 3

Menu invariance and preference transitivity. a. One neuron encoding the offer value. In this experiment, monkeys chose between 3 juices (A, B and C) offered pairwise. The three panels refer, respectively, to trials A:B, B:C and C:A. In each panel, the x-axis represents different offer types, black circles indicate the behavioral choice pattern and red symbols indicate the neuronal firing rate. This neuron encodes the variable offer value C independently of whether juice C is offered against juice B or juice A. In trials A:B, the cell activity is low and not modulated. b. Linear encoding. Same neuron as in 3a, with the firing rate (y-axis) plotted against the encoded variable (x-axis) separately for different trial types (indicated by different symbols, see legend). c. Value transitivity. For each juice pair X:Y, the relative value n_XY is measured from the indifference point. The three relative values satisfy transitivity if (in a statistical sense) n_AB * n_BC = n_AC. In this scatter plot, each circle indicates one session (± s.d.) and the two axes indicate, respectively, n_AB * n_BC and n_AC. Data lie along the identity line, indicating that subjective values measured in this experiment satisfy transitivity. Choices based on a representation of value that is menu invariant are necessarily transitive. Adapted from Padoa-Schioppa and Assad (2008), Nature Neurosci (Nature Publishing Group).

Figure 4

Range adaptation in the valuation system. a. Model of neuronal adaptation. The cartoon depicts the activity of a value-encoding neuron adapting to the range of values available in different conditions. The x-axis represents value, the y-axis represents the firing rate and different colors refer to different value ranges. In different conditions, the same range of firing rates encodes different value ranges. b. Neuronal adaptation in the OFC. The figure illustrates the activity of 937 offer value responses. Each line represents the activity of one neuron (y-axis) plotted against the offer value (x-axis). Different responses were recorded with different value ranges (see color labels). While activity ranges vary widely across the population, the distribution of activity ranges does not depend on the value range. c. Population averages. Each line represents the average obtained from neuronal responses in 4b. Adaptation can be observed for any value, as average responses are separated throughout the value spectrum. Similar results were obtained for neurons encoding the chosen value. Adaptation was also observed for individual cells recorded with different value ranges. Adapted from Padoa-Schioppa (2009) J Neurosci (Soc for Neurosci, with permission).

Figure 5

Effects of selective devaluation. In the training phase of this study, rats learned to perform a task (lever press or chain pull) to obtain a reward (food pellet or starch, in a counterbalanced design). Before testing, animals were selectively satiated with one of the two foods (devaluation). They were then tested in extinction. Thus their performance, measured in actions per minute (y-axis), dropped over time (x-axis) for either food. Critically, the performance for the devalued food (filled symbols) was consistently below that for the control food (empty symbols). Adapted from Balleine and Dickinson (1998), Neuropharmacology (Elsevier, with permission).

Figure 6

Action values signals downstream of the decision. a. Activity profiles from OFC, lateral prefrontal cortex (PFC), supplementary eye fields (SEF), frontal eye fields (FEF), premotor cortex (PM), supplementary motor area (SMA) and muscle electromyographic activity (EMG). For each brain region, black and grey traces refer, respectively, to trials with high and low value. Left and right panels refer to saccades towards, respectively, the preferred and anti-preferred directions. For each area, the overall difference between the activity observed in the left and right panels (highlighted in b1) can be interpreted as encoding the action. The difference between the black and grey traces (highlighted in b2) is a value modulation. b. Summary of action value signals. The top panel (b1) highlights the encoding of possible actions (contraversive and ipsiversive for blue and red bars, respectively). The bottom panel highlights value modulations (positive and negative encoding for blue and red bars, respectively). Action encoding is minimal in the OFC but significant in all motor areas. In contrast, value modulation is significant both in the OFC and in motor areas. Strikingly, there is a strong value modulation also in the EMG (bottom panels in 6a). Muscles certainly do not contribute to economic choice – a clear example of action value unrelated to the decision. Thus value modulations in the motor areas – which ultimately control the motor output – are most likely related to value modulations in the EMG, not to the decision process per se. Adapted from Roesch and Olson (2003, 2005), J Neurophysiol (Am Physiol Soc, with permission).