Serotonin and dopamine: unifying affective, activational, and decision functions

Abstract

Serotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcement learning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance, compared with DA, we have a much less specific understanding about the mechanisms by which serotonin affects behavior. Here, we draw on recent work on computational models of dopaminergic function to suggest a framework by which many of the seemingly diverse functions associated with both DA and serotonin-comprising both affective and activational ones, as well as a number of other functions not overtly related to either-can be seen as consequences of a single root mechanism.

Figure 1

Graphic depiction of the core computational concepts outlined in this article, and their consequences for the functional domains of response vigor, time discounting, switching, and risk sensitivity. (a) Phasic time series of rewards and punishments, together with tonic signals consisting of the slow running average of each. We associate these average reward and punishment signals with tonic dopamine and serotonin, respectively. The difference between them (the overall average outcome), in black, is expected to control (b–e) a number of aspects of decision making. These subfigures illustrate how different decision-related calculations are impacted when the average outcome increases from less rewarding (solid lines and bars) to more rewarding (dashed lines and hollow, offset bars). Black arrows indicate the directions of change, and asterisks indicate preferred options. Rewards are illustrated in blue and costs or punishments in red. (b) The choice of how quickly to perform an instrumental behavior can be guided by trading off the ‘energetic cost' of performing it more quickly against the ‘opportunity cost' of the time spent. When the average outcome improves, the opportunity cost grows more quickly with time spent, and the point that minimizes the total cost (the optimal choice of response speed, asterisk) shifts to the left, favoring quicker responding. (c) The choice between a small reward soon (top) or a large reward later (bottom) can be guided by balancing the rewards against the opportunity costs of the delays. When the average outcome improves, the opportunity cost of the longer delay weighs more heavily, shifting the preferred choice from patient to impatient. (d) Learning about the value of an action can be guided by comparing the reward obtained with the average outcome expected. When the average outcome improves, the comparison term can drive the net reinforcement negative, and instead of reinforcing the action (favoring staying) it will be punished, favoring switching. (e) Preference over prospective options involving risk may depend on whether the outcomes are net gains or losses, when measured relative to a reference point. Humans and other animals tend to be risk averse when considering prospects involving (relative) gains and risk prone when considering relative losses. Here, if the average outcome is taken as the reference point, the choice between a sure win (top, safe) and a 50/50 small or large win (top, risky) and a sure win (bottom, safe), shifts from the gain domain to the loss domain when the average outcome improves, leading to a shift in preference from the safe option to the risky one.

Figure 2

Preliminary empirical evidence for a role for serotonin in the interaction between vigor and negative reward signals. (a) Left panel: experimental paradigm employed by Crockett et al (2009). In the reward–go condition, subjects received large rewards for correct go responses and small rewards for correct nogo responses. In the punish–go condition, subjects received large punishments for incorrect nogo responses and small punishments for incorrect go responses. The complementary reward–nogo and punish–nogo conditions are not depicted here. Right panel: effect of tryptophan depletion on correct go reaction times in punished conditions relative to rewarded conditions. Tryptophan depletion abolished punishment-induced slowing. Reproduced with permission from Crockett et al (2009). (b) Effect of tryptophan depletion on response vigor as a function of reward likelihood (Cools et al, 2005). In this experiment, subjects responded as fast as possible to a target that was preceded by one of three reward cues, signaling 10, 50, or 90% reward likelihood. After placebo, subjects responded more slowly in response to low reward cues relative to high reward cues. Tryptophan depletion speeded reaction times in response to cues signaling low reward likelihood (depletion × reward cue interaction; P=0.009). Error bars represent standard errors of the mean. (c) Three types of modulation of activity of primate dorsal raphé neurons during a memory-guided saccade task. Histograms are aligned to fixation point onset (left), target onset (middle), and outcome onset (right). Lines indicate mean firing rate of all trials (black), large-reward trials (red), and small-reward trials (blue). Black asterisks indicate significant difference in activity during the 500–900 ms after fixation point onset compared with a 400 ms prefixation period (P<=0.005, rank-sum test). Red and blue asterisks indicate significant difference between two reward conditions during 150–450 ms after target onset, go onset, or outcome onset. Top panel: example of a neuron that increased its activity during the task and fired more for large- than small-reward trials after target onset and outcome onset. Middle panel: example of a neuron that decreased its activity during the task and fired more for small- than large-reward trials after target onset and outcome onset. Bottom panel: example of a neuron that did not change its activity during the task and did not show a significant reward effect after target onset and outcome onset. Panel c reproduced with permission from Bromberg-Martin et al (2010).