Serotonin, inhibition, and negative mood

Abstract

Pavlovian predictions of future aversive outcomes lead to behavioral inhibition, suppression, and withdrawal. There is considerable evidence for the involvement of serotonin in both the learning of these predictions and the inhibitory consequences that ensue, although less for a causal relationship between the two. In the context of a highly simplified model of chains of affectively charged thoughts, we interpret the combined effects of serotonin in terms of pruning a tree of possible decisions, (i.e., eliminating those choices that have low or negative expected outcomes). We show how a drop in behavioral inhibition, putatively resulting from an experimentally or psychiatrically influenced drop in serotonin, could result in unexpectedly large negative prediction errors and a significant aversive shift in reinforcement statistics. We suggest an interpretation of this finding that helps dissolve the apparent contradiction between the fact that inhibition of serotonin reuptake is the first-line treatment of depression, although serotonin itself is most strongly linked with aversive rather than appetitive outcomes and predictions.

Figure 1. Markov Models of Thought

The abstract state space is divided into the four blocks shown. The right two, O℘ ₊ and O℘ ₋, are associated with direct affective values r(s) (inset histograms); the left two, I℘ ₋ and I℘ ₊, are internal. Transitions between (belief) states are determined by actions (thoughts). We initially focus on a fixed policy, leading to the transition between states shown in the figure: states in each internal block I℘ ₊ and I℘ ₋ preferentially connect with each other and their respective outcome states O℘ ₊ and O℘ ₋. However, each state has links to states in the other block. The model is approximately balanced as a whole, with an equal number of positive and negative states.

Figure 2. Probability of Continuing a Train of Thoughts

For values V(s) > 0, thoughts are continued with probability 1. Conversely, when the state s has negative value, the probability of continuation drops of as an exponential function of the value. The rate of the exponential is set by α _5HT.

Figure 3. Learning with Behavioral Inhibition

(A,B) With α _5HT = 0, for one particular learning run, the values V _est match their true values V _true (inferred through dynamic programming) under an equal-sampling exploration policy (A), and trains of thought end in terminal states O℘ ₋, O℘ ₊ equally often as a function of their actual outcomes (B) (the red line is the regression line). (C,D) With α _5HT = 20, negative V values are poorly estimated (since exploration is progressively inhibited for larger α _5HT), and the more negative the value of the outcome, the less frequently that outcome gets visited over learning (D). Importantly, there is an optimistic underestimate of the negative value of state. (E) The root mean squared error (averaging over 20 runs) for states with positive (dotted) and negative V _true values as a function of α _5HT. The effect of the sampling bias is strikingly apparent, preventing accurate estimates mainly of the negatively valued states. (F) Average reward received during learning as a function of α _5HT—the benefits of behavioral inhibition are apparent.

Figure 4. Reduced Inhibition

These graphs show statistics of the effect of learning V values with α _5HT = 20, and then suffering from reduced serotonin α _5HT < 20 during sampling of thoughts. For a given thought environment, these are calculated in closed form, without estimation error. (A) As is also evident in Figure 3F, the average affective return is greatly reduced from the value with α _5HT = 20; in fact, for the extreme value of α _5HT = 0, it becomes slightly negative (reflecting a small sample bias in the particular collection of outcomes). (B,C) Normalized outcome prediction errors at the time of transition to O℘ ₊ (B) or O℘ ₋ (C) for α _5HT = 20 against α _5HT = 0. These reflect the individual probability that each terminal transition goes to r(s) from V(s′) for s ∈ O℘ and s′ ∈ I℘, including all the probabilistic contingencies of termination, etc. They are normalized for the two values of α _5HT. Terminations in O℘ ₊ are largely unaffected by the change in inhibition; terminations in O℘ ₋ with negative consequences have greatly increased negative prediction error.

Figure 5. Reward Seeking and Recall Bias

Both plots are in the same form as Figure 4A, showing the percentage utilities compared with the standard learning case α _5HT = 20, as a function of α _5HT (the emboldened blue curve is exactly that in Figure 4A). (A) Given a mood-dependent bias on the starting state, with , the plots show the consequences of various values of β. Negative β, favoring low value states, leads to substantially negative average outcomes. (B) Instrumental control of action choice, a putative model of dopaminergic effects, can also either exacerbate or improve the outcomes, depending on the value of the parameter θ governing a softmax choice of actions.