Mood as Representation of Momentum

Abstract

Experiences affect mood, which in turn affects subsequent experiences. Recent studies suggest two specific principles. First, mood depends on how recent reward outcomes differ from expectations. Second, mood biases the way we perceive outcomes (e.g., rewards), and this bias affects learning about those outcomes. We propose that this two-way interaction serves to mitigate inefficiencies in the application of reinforcement learning to real-world problems. Specifically, we propose that mood represents the overall momentum of recent outcomes, and its biasing influence on the perception of outcomes 'corrects' learning to account for environmental dependencies. We describe potential dysfunctions of this adaptive mechanism that might contribute to the symptoms of mood disorders.

Keywords: Mood; decision making; reinforcement learning.

Figure 1

The Effect of a Monetary Outcome on Mood and on Subsequent Neural and Behavioral Responses to Rewards. (A) Experimental design of . To manipulate mood, a one-shot wheel-of-fortune (WoF) draw was held in between games, resulting in a gain or loss of $7. Game 1 and Game 2 involved different sets of slot machines with similar reward probabilities, and participants learned about the machines by trial and error. In the Test phase, participants chose between slot machines from Game 1 and Game 2. (B) In participants who reported high emotional instability, the WoF outcome affected self-reported mood (left, n = 28 per group; 1 is maximally happy and −1 is maximally unhappy) and striatal BOLD response to reward measured by fMRI (middle, n = 13 per group) during Game 2 as compared to Game 1 (shown are t values). In the test phase, those participants who experienced a WoF win preferred Game 2 machines, which they had played while in a better mood. By contrast, participants who had experienced a WoF loss preferred Game 1 machines, which they played before the WoF draw (right, n = 28 per group; 1 indicates complete preference for Game 2 machines). * P < 0.05, ** P < 0.001. Adapted from .

Figure 2

Schematic of Possible Mood Dysfunctions. (A) (Top) Given a similar rate of learning in response to positive and negative outcomes, an environment in which positive and negative outcomes are equally likely leads to neutral expectations and a neutral mood on average. (Bottom) A lower rate of learning from negative outcomes leads to optimistic expectations and therefore larger negative prediction errors and persistent negative mood, a symptom seen in major depressive disorder. (B) Escalatory positive-feedback dynamics might turn mood into a ‘self-fulfilling prophecy’, leading to emotional instability, a major symptom of bipolar disorder. Positive surprises improve mood, biasing perception of outcomes upwards, thereby increasing the frequency and magnitude of further positive surprises. Optimistic expectations develop owing to the biased perception of outcomes. Mood stabilizes once expectations catch up with perceived outcomes, but subsequent outcomes, whose perception in now unbiased, then tend to fall short of optimistic expectations. Thus, negative surprises follow, thereby diminishing mood and biasing perception of outcomes downward. Similar positive-feedback dynamics then engender pessimistic expectations, setting the stage for the cycle to repeat, oscillating between good and bad mood indefinitely even if there are no changes in the actual distribution of outcomes.

Figure I

Subjective Well-Being, Model Predictions, and Neural Activity. (A) Happiness ratings during a probabilistic reward task for two example participants. The model predicts ratings based on the past history of expectations and reward prediction errors (RPEs) resulting from those expectations. (B) BOLD activity in the ventral striatum measured by fMRI was correlated with subsequent happiness ratings, consistent with striatal representation of RPEs contributing to changes in mood. Adapted from .

Figure I

Probabilistic Kalman–Filter Models of the Environment. The reward outcome r at time-step t is sampled from a normal distribution whose mean $v_t^s$ is specific to the current state. (A) For a particular state, depicted here, changes in the mean follow a random walk with normally distributed noise. (B) A general environmental factor affects multiple states. At each time-step t, a general factor g_t is sampled from a normal distribution whose mean is zero, and is then added to multiple state means ($v_t^s$). (C) Changes in reward follow an underlying momentum. The mean reward $v_t^s$ of a state is sampled from a normal distribution whose mean is the sum of the previous mean $v_{t-1}^s$ and the current momentum m_t. Changes in momentum follow a random walk.