The modulation of acute stress on model-free and model-based reinforcement learning in gambling disorder

Abstract

Background and aims: Experiencing acute stress is common in behavioral addictions such as gambling disorder. Additionally, like most substance-induced addictions, aberrant decision-making wherein a reactive habit-induced response (conceptualized as a Model-free [MF] in reinforcement learning) suppresses a flexible goal-directed response (conceptualized as a Model-based [MB]) is also common in gambling disorder. In the current study we investigated the influence of acute stress on the balance between habitual response and the goal-directed system.

Methods: A sample of N = 116 problem gamblers (PG) and healthy controls (HC) performed an acute stress task - the Socially Evaluated Cold pressure task (SECPT) - or a control task. Self-reported stress and salivary cortisol were collected as measures of acute stress. Following the SECPT, participants performed the Two-Step Markov Task to account for the relative contribution of MB and MF strategies. Additionally, verbal working memory and IQ measures were collected to account for their mediating effects on the orchestration between MB/MF and the impact of stress.

Results: Both groups had comparable baseline and stress-induced cortisol response to the SECPT. Non-stressed PG displayed lower MB learning than HC. MANOVA and regression analyses showed a deleterious effect of stress-induced cortisol response on the orchestration between MB and MF learning in HC but not in PG. These effects remained when controlling for working memory and IQ.

Discussion and conclusions: We found an abnormal pattern of modulation of stress on the orchestration between MB and MF learning among PG. Several interpretations and future research directions are discussed.

Keywords: Two-Step Markov Task; gambling disorder; model-based; model-free; reinforcement learning; stress.

Fig. 1.

Experimental protocol. After filling out the informed consent form, participants underwent the Raven's Standard Progressive Matrices Test (RSPM) and OSPAN task. The first cortisol measurement (C1) was collected 10 min after the arrival. The second (C2) was collected right after the dual-step Markov Task (RL task) instruction along with the first self-reported evaluation (VAS1). The second VAS was taken right after the stress induction (SECPT)/control (WPT) procedure, while the third cortisol measurement (C3) was collected 10 min after the procedure. The fourth measurement (C4) was collected right after the Two-Step Markov Task (RL task). Participants finished by filling out several questionnaires

Fig. 2.

A. Two-step decision task. (First step) Participants chose between the two images, leading preferentially to a green or a blue screen, according to fixed probabilities. (Second step) Subjects chose between the two images linked to probabilities to win money. Those probabilities slowly changed with time and differed according to the background color. B. Second step's changes in the probability of reward. C. Trial's design. D. Theoretical decision pattern according to a pure MF strategy and to a pure MB strategy

Fig. 3.

Cortisol and self-reported measures for the group that underwent the cold-pressor task (SECPT) and the Warm-Pressor Task (WPT). Graphs show mean values ±SE. P-values of the effect of interaction between time and procedure are reported * P < 0.05, ** P < 0.01, *** P < 0.001

Fig. 4.

Probability to maintain a first step choice in problem gamblers (PG) and healthy controls (HC) in trial following: a reward and a common transition (PRC), a reward and a rare transition (PRR), a loss and a common transition (PUC), as well as a loss and a rare transition (PUR). Graphs show mean values ±2 SE

Fig. 5.

Mean of the computational parameters among responder pathological gamblers (PG S), non-responder problem gamblers (PG NS), responder controls (HC S), and non-responder controls (HC NS). Graphs show mean values ±2 SE

Fig. 6.

Effect of the cortisol increase on ω-parameter in problem gamblers (PG) and healthy controls (HC), as determined by the computational model. Each continuous score was standardized, 95% IC is displayed around each regression line