How Accumulated Real Life Stress Experience and Cognitive Speed Interact on Decision-Making Processes

Abstract

Rationale: Advances in neurocomputational modeling suggest that valuation systems for goal-directed (deliberative) on one side, and habitual (automatic) decision-making on the other side may rely on distinct computational strategies for reinforcement learning, namely model-free vs. model-based learning. As a key theoretical difference, the model-based system strongly demands cognitive functions to plan actions prospectively based on an internal cognitive model of the environment, whereas valuation in the model-free system relies on rather simple learning rules from operant conditioning to retrospectively associate actions with their outcomes and is thus cognitively less demanding. Acute stress reactivity is known to impair model-based but not model-free choice behavior, with higher working memory capacity protecting the model-based system from acute stress. However, it is not clear which impact accumulated real life stress has on model-free and model-based decision systems and how this influence interacts with cognitive abilities. Methods: We used a sequential decision-making task distinguishing relative contributions of both learning strategies to choice behavior, the Social Readjustment Rating Scale questionnaire to assess accumulated real life stress, and the Digit Symbol Substitution Test to test cognitive speed in 95 healthy subjects. Results: Individuals reporting high stress exposure who had low cognitive speed showed reduced model-based but increased model-free behavioral control. In contrast, subjects exposed to accumulated real life stress with high cognitive speed displayed increased model-based performance but reduced model-free control. Conclusion: These findings suggest that accumulated real life stress exposure can enhance reliance on cognitive speed for model-based computations, which may ultimately protect the model-based system from the detrimental influences of accumulated real life stress. The combination of accumulated real life stress exposure and slower information processing capacities, however, might favor model-free strategies. Thus, the valence and preference of either system strongly depends on stressful experiences and individual cognitive capacities.

Keywords: chronic stress; cognitive speed; decision making; model-based learning; model-free learning; real-life events.

Figure 1

Digit Symbol Substitution Test (DSST, Wechsler, 1997): the DSST consists of a code table displaying pairs of digits and symbols, and rows of double boxes with a digit in the top box and an empty space in the bottom box. The task for the subject is to use the code table to determine the symbol associated with each digit, and to write as many symbols as possible in the empty boxes below each digit within 120 s.

Figure 2

(A) Structure of the Two-Step Task. (B) Simulated data of a pure model-free vs. a pure model-based decision-maker. Model-free and model-based strategies predict distinct response patterns on the first stage. In model-free decisions, first stages should be repeated whenever the outcome of the previous trial was rewarded, whereas they should not be repeated whenever choices were unrewarded. Therefore model-free decisions predict a main effect of reward on first stage repetition of the subsequent trial. In model-based decisions, the individual takes transition frequencies into account. Thus, for instance, when a trial from the rare transition frequency ended up in reward, the individual knows that in order to obtain reward in the next trial he/she should actually switch to the opposing first stage stimuli, because this one has a higher probability of ending up at the specific second stage stimulus pair that is now associated with high probability of reward. (C) Across all subjects (N = 95), model-free and model-based scores were significantly positive (as indicated with the *** both p < 0.0001), suggesting that subjects showed a mixture of model-free and model-based choice strategies.

Figure 3

(A) Effects of stress and cognitive speed on model-free vs. model-based control: subjects with low cognitive speed display an increase of model-free but a decrease of model-based behavior with increasing stress exposure, whereas subjects with high cognitive speed display an increase of model-based but a decrease of model-free behavior with increasing stress exposure. (B) The differential association of cognitive speed with model-free and model-based control was particularly evident in subjects who had experienced high stress (significant effects indicated with *), whereas there was no such effect in subjects who had experienced low stress.