Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

Abstract

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

Keywords: anxiety; avoidance; computational modeling; reinforcement learning; uncertainty.

Fig. 1

Types of uncertainty. A. Relationship among uncertainty types. B. Schematic of levels of types of uncertainty during a reversal learning task. Initially, outcome 1 is reinforced more than outcome 2 (reinforcement represented by dots in top part of figure, where x-axis indicates trials). Halfway through, these contingencies reverse, and a second reversal occurs ~75% of the way through. Approximate values of each type of uncertainty at different points in the task are represented in the lower part of the figure. Irreducible uncertainty is initially high before converging on a value close to the true noise level; this form of uncertainty only increases slightly after the changepoints. Estimation uncertainty is initially high, reflecting uncertainty about contingencies. This uncertainty reduces while contingencies are learned but then increases after changepoints, reflecting the need to relearn contingencies once they have changed. As changepoints become more frequent and unexpected uncertainty increases, corresponding increases in learning rate lead to faster reductions in estimation uncertainty between changepoints. Unexpected uncertainty is also initially high and decreases while contingencies are stable. After each changepoint, this uncertainty increases and stays higher, reflecting the increased possibility of additional changepoints

Fig. 2

Markov decision processes. A. MDP formulation of an instrumental learning paradigm or bandit task. Each trial has two stimuli: one with a 75% chance of leading to reward and a 25% chance of leading to no reward; the other stimulus has the opposite probabilities. This MDP has three states: the state prior to choice (s₃), where the agent is presented with the two stimuli; two outcome, or terminal, states, s₂ and s₁; and two actions, choosing one stimulus (a₁) and choosing the other (a₂). Transitioning from state s₃ to the two terminal states s₁ and s₂ have rewards of 1 and 0, respectively. There is a transition probability for each action-transition pair: Pa₁(s₃, s₁) (the probability of transitioning from state s₃ to state s₁ given action a₁) is 0.75, Pa₁(s₃, s₂) is 0.25, Pa₂(s₃, s₁) is 0.25, and Pa₂(s₃, s₂) is 0.75; these probabilities re-express the 75%/25% and 25%/75% outcome contingencies for the stimuli. In this MDP, there is one choice: select either a₁ or a₂. B. MDP formulation of an instrumental learning paradigm with two steps to the outcome. Compared to the MDP in panel A, this MDP has another state (s₄) with another set of stimuli to choose between that lead to reward, and a state (s₅) where one chooses between stimuli that probabilistically lead to states s₄ or s₃. C. MDP of a set of possible states and actions for a person with social anxiety going to a party. This example illustrates a subset of potential states and actions (e.g., another possible state after the “make a joke” action is that the group laughs at your joke, but then you accidentally sneeze on someone, which also leads to the core fear of negative social evaluation). Note that there are three terminal states: staying in a conversation (person’s goal, small positive reward), returning home without engaging in conversation (total avoidance, no reward), and being excluded (feared outcome, large negative reward). D. Abstraction of the MDP in Panel C. E. First stage of Pavlovian learning as proposed by (Moutoussis et al., 2008) and (Maia, 2010). Values propagate from the terminal states to other states and actions through learning. Values inside each state (represented by boxes) represent the learned value of each state, whereas values next to arrows represent the change in value when transitioning between those states. F. Second stage of instrumental learning as proposed by (Moutoussis et al., 2008) and (Maia, 2010). Actions are taken with frequencies, denoted by line thickness, based on the values acquired through Pavlovian learning in the first stage

Fig. 3

Effects of uncertainty on explore/exploit decisions within a Markov decision process. Top, effect of each type of uncertainty on exploration in appetitive and aversive environments. In rewarding environments, irreducible uncertainty decreases exploration, while estimation uncertainty increases; effects are opposite in aversive environments. Effects of unexpected uncertainty have been less studied, so are qualified with ?, but appear to increase exploration in appetitive contexts and so may decrease exploration in aversive contexts. Bottom, possible levels of uncertainty under different circumstances and effects on exploration in aversive environments. As anxiety is hypothesized to primarily affect uncertainty estimation in aversive environments, explore-exploit behavior should show few differences with anxiety when learning about reward.

Fig. 4

Illustration of changes in value and types of uncertainty with anxiety, avoidance, and successful exposure therapy, and effects on changes in value and uncertainty on avoidance. Example scenario, from the social anxiety MDP in Fig. 2, illustrates a choice between an avoidance behavior, leaving the group to stand by the snack table, and a nonavoidance behavior, making a joke during the conversation. In normative behavior, the nonavoidance behavior has higher value but also greater irreducible uncertainty (e.g., there is a greater risk that making a joke will go poorly and lead to negative outcomes compared to leaving to get a snack). With high anxiety, before developing avoidance behaviors, uncertainty is miscalculated as greater estimation and/or unexpected uncertainty rather than irreducible uncertainty, leading to greater tendency to avoid the more uncertain stimulus and choose the safe avoidance behavior. Engaging in avoidance behaviors increases the value of the avoidance vs. nonavoidance behavior (due the temporal difference learning processes illustrated in Fig. 2E and F). Additionally, greater experience with outcomes stemming from the avoidance behavior reduces uncertainty associated with the chosen behavior while the uncertainty associated with the unchosen, nonavoidance behavior is not reduced. Both of these processes increase the tendency to avoid. Initial exposure sessions begin to correct uncertainty associated with the nonavoidance behavior through experience with that choice’s outcomes. Avoidance is decreased relative to pre-exposure behavior but is still higher than normative behavior. After repeated exposures, the relative values of avoidance vs. nonavoidance behavior normalize along with further corrections in uncertainty calculations. As a result, the tendency to avoid becomes similar to normative behavior