Phasic dopamine signals: from subjective reward value to formal economic utility

Abstract

Although rewards are physical stimuli and objects, their value for survival and reproduction is subjective. The phasic, neurophysiological and voltammetric dopamine reward prediction error response signals subjective reward value. The signal incorporates crucial reward aspects such as amount, probability, type, risk, delay and effort. Differences of dopamine release dynamics with temporal delay and effort in rodents may derive from methodological issues and require further study. Recent designs using concepts and behavioral tools from experimental economics allow to formally characterize the subjective value signal as economic utility and thus to establish a neuronal value function. With these properties, the dopamine response constitutes a utility prediction error signal.

Figure 1

Subjective value coding. (a) Graded impulse responses of dopamine neurons to different reward-predicting stimuli (blackcurrant juice and mashed banana mix). Arrows indicate subjective behavioral preferences; ~ indifferent. imp/s: firing rate. From Lak et al. [4••]. (b) Incorporation of negative aversive value into the common currency dopamine signal. The impulse response to reward juice alone (black) is reduced when an aversive salt or bitter solution is delivered together with the juice. +imp/s: firing rate subtracted from baseline firing. From Fiorillo [14•]. (c) Influence of risk on dopamine impulse responses to stimuli predicting differentially two liquid rewards (blue, blackcurrant juice; green, orange juice). Top: stimuli predicting binary, equiprobable, risky gambles (double horizontal bars) and safe rewards (single bars) with identical mean amounts. Vertical bar height indicates reward amount. Bottom: corresponding neuronal responses, closely following subjective values inferred from behavioral preferences shown above. Arrows indicate response increases with risky over safe rewards in risk seeking animal. From Lak et al. [4••]. (d) Differential, risk-attitude dependent influences of risk on voltammetric dopamine responses in rat nucleus accumbens to cue lights. Equiprobable risk reduces response in risk avoiders (top) but increases response in risk seekers (bottom). Mean reward is identical for gamble and safe reward (1 pellet). From Sugam et al. [9••].

Figure 2

Temporal discounting. (a) Decreasing impulse responses of dopamine neurons to stimuli differentially predicting increasing reward delays (red), corresponding to subjective value decrements assessed in intertemporal choices (blue). Y-axis shows behavioral value and neuronal responses in % of reward amount at 2 s delay (0.56 ml). From Kobayashi and Schultz [25]. (b) Lower voltammetric dopamine responses in rat nucleus accumbens to visual cues specifying longer reward delay with identical effort. From Day et al. [26].

Figure 3

Voltammetric dopamine responses under different effort loads. (a) High effort (lever press FR16) compared to low effort (FR1) decreases dopamine response in rat nucleus accumbens (from 70 nM to 50 nM). From Day et al. [26]. (b) Behavioral preference for low effort–low reward (LL) is not associated with higher voltammetric dopamine response in 3 nM range. From Hollon et al. [29•].

Figure 4

Utility prediction error coding by dopamine neurons. (a) Nonlinear impulse response to unpredicted juice reward generating positive reward prediction errors. Red: nonlinear utility function. Black: dopamine responses in same monkey. Blue: linear increases in unpredicted reward, generating linearly increasing positive reward prediction errors. The linearly increasing reward prediction errors are coded by dopamine neurons as nonlinearly increasing utility prediction errors, thus showing a neuronal utility signal. (b) Nonmonotonic utility prediction error signal with constant-risk gambles. The higher gamble outcomes constitute positive utility prediction errors (Δu) whose magnitudes depend on the local slope of the utility function (top). Dopamine responses follow the nonmonotonic changes of the positive utility prediction error (Δu). a and b from Stauffer et al. [18••].