Dopamine signals for reward value and risk: basic and recent data

Abstract

Background: Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli.

Methods: The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks.

Results: Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward.

Conclusions: Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

Figure 1

Phasic activations of neurophysiological impulse activity of dopamine neurons. A: Phasic activations following primary rewards. B: Phasic activations following conditioned, reward predicting stimuli. C: Top: Lack of phasic activation following primary aversive air puff. Bottom: substantial activating population response following conditioned aversive stimuli when stimulus generalization by appetitive stimuli is not ruled out; grey: population response to conditioned visual aversive stimulus when appetitive stimulus is also visual; black: lack of population response to conditioned visual aversive stimulus when appetitive stimulus is auditory. D: Phasic activations following physically intense stimuli. These activations are modulated by the novelty of the stimuli but do not occur to novelty per se. E: Left: Shorter and smaller activations followed frequently by depressions induced by unrewarded control stimuli (black) compared to responses following reward predicting stimuli (grey). Right: Activations to delay predicting stimuli show initial, poorly graded activation component (left of line) and subsequent, graded value component inversely reflecting increasing delays (curves from top to bottom). Time scale (500 ms) applies to all panels A-E. Data from previous work [29,31-33,43,59].

Figure 2

Sustained activations related to risk. The risk response occurs during the stimulus-reward interval (arrow) subsequently to the phasic, value related activation to the stimulus (triangle). The inset, top right, shows that risk (ordinate) varies according to an inverted U function of reward probability (abscissa) (Data from previous work [18].