A neuronal prospect theory model in the brain reward circuitry

Abstract

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals' choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

Fig. 1. Cued lottery task and monkeys’ choice behavior.

a A Sequence of events in the choice trials. Two pie charts representing available options were presented to the monkeys on the left and right sides of the screen. The monkeys chose either of the targets by fixating on the side where they appeared. b Frequency with which the target on the right side was selected for the expected values of the left and right target options. c Sequence of events in single-cue trials. d AIC values are estimated based on the four standard economic models to describe the monkey’s choice behavior: EV, EU, PT1, and PT2. See the Methods section for details. e Estimated utility functions in the best-fit model PT2. f Estimated probability-weighting functions in the best-fit model PT2. Images in panels a-c were created by the authors and previously published in Neural Population Dynamics Underlying Expected Value Computation. Hiroshi Yamada, et al.. https://creativecommons.org/licenses/by/4.0/.

Fig. 2. Neural coding of probability and magnitude of rewards in the four brain regions.

a Illustration of neural recording areas based on coronal magnetic resonance images. b Example activity histogram of a DS neuron modulated by probability and magnitude of rewards with positive regression coefficients during the single-cue task (P + M + type). The activity aligned to the cue onset is represented for three different levels of probability (0.1–0.3, 0.4–0.7, and 0.8–1.0) and magnitude (0.1–0.3 mL, 0.4–0.7 mL, and 0.8–1.0 mL) of rewards. Gray hatched time windows indicate the 1-s time window used to estimate the neural firing rates shown in f and g. Raster grams are shown below. c–e similar to b, but for VS, cOFC, and mOFC neurons. f Plot of the neural firing rates during the 1-s time window in b for ten levels of probability and magnitude of rewards. The firings are normalized by the maximum firing rates. P and M indicate the probability and magnitude of rewards, respectively. g Color map of the neural firing rates during the 1 s time window in b for ten levels of probability and magnitude of rewards. Average smoothing was made between neighboring pixels. h Percentage of neurons modulated by probability and magnitude of rewards in the four core reward brain regions. Gray indicates activity showing positive regression coefficients for probability and magnitude of rewards (P + M + type). Black indicates activity showing the negative regression coefficients for probability and magnitude (P-M- type). Images in panels a were created by the authors and previously published in Neural Population Dynamics Underlying Expected Value Computation. Hiroshi Yamada, et al.. https://creativecommons.org/licenses/by/4.0/.

Fig. 3. Neural models of economic decision theory.

Schematic depiction of predicted neuronal responses R defined by the four economic models that represent the expected value (a, EV), expected utility (b, EU), prospect theory one-parameter Prelec (c, PT1), and two-parameter Prelec (d, PT2). Model equations are presented in each plot. R was plotted against the probability (p) and magnitude (m) of the rewards. b, g, α, γ, and δ are the free parameters. g and b are the gain and intercept parameters, respectively. α represents the curvature of u(m). δ and γ represent the probability weighting functions. For these schematic drawings, the following values for the free parameters were used: b, g, α, γ, and δ were 0 spk s⁻¹, 1, 0.6, 2, and 0.5, respectively, for all four Figs. See the Methods section for more details.

Fig. 4. Prospect theory best explained neural firing rates in the reward circuitry.

a Plot of an example activity of the DS neuron in Fig. 2b against the probability (p) and magnitude (m) of rewards. To draw the 3D curvature (left) and contour lines (right), the neighboring pixels were averaged and smoothed. b AIC values against the proportion of variance explained are plotted in each model for the example neuron in a. c A 3D histogram (left) and contour lines (right) predicted from the best-fit PT2 model in a. The activity of the example neuron in a is shown on the right color map. Contour lines are shown for every 10% change in the fit model. d u(m) and w(p) estimated in the best-fit model PT2 for the neural activity in a. e Probability density of the estimated AIC difference of the three models against the EV (simplest) model. The plots display the mean values. n represents the number of neuronal signals that showed both positive and negative regression coefficients for the probability and magnitude of the rewards.

Fig. 5. Neuronal clusters categorized by the fitted parameters according to the prospect theory model.

a Plots of all five parameters estimated in DS, VS, and cOFC neurons. g, b, α, δ, and γ were plotted. b Cumulative plot of the proportion of variance explained by PCA is shown against principal components PC1–PC5. c Cumulative plot of the percentage of activity categorized into five clusters in each brain region. d Response R (model output) in the first three predominant clusters is plotted. The 3D curvature, contour lines with color maps, u(m), and w(p) are plotted using the mean values of each parameter in each cluster. To draw the 3D curvature (first column) and contour lines (second column), R was normalized to the maximal value.

Fig. 6. A simple network model reconstructs the subjective decision statistics in monkeys.

a Five neural clusters detected by PCA in the reward circuitry. To draw the 3D curvature, R was normalized to the maximal value. The subjective expected value functions (SEV) for the left and right target options are defined as the linear summation of the five clusters (see Methods). The choice was simulated as a sigmoid function of the SEV’s signal difference. b Frequency with which the target on the right side was selected by a computer simulation based on the network shown in a. c u(m) and w(p) estimated from the simulated choice data in b are plotted. The dotted lines indicate the actual functions u(m) and w(p) of the monkeys, as shown in Fig. 1e and f, respectively.