The influence of depression symptoms on exploratory decision-making

Abstract

People with symptoms of depression show impairments in decision-making. One explanation is that they have difficulty maintaining rich representations of the task environment. We test this hypothesis in the context of exploratory choice. We analyze depressive and non-depressive participants' exploration strategies by comparing their choices to two computational models: (1) an "Ideal Actor" model that reflectively updates beliefs and plans ahead, employing a rich representation of the environment and (2) a "Naïve Reinforcement Learning" (RL) model that updates beliefs reflexively utilizing a minimal task representation. Relative to non-depressive participants, we find that depressive participants' choices are better described by the simple RL model. Further, depressive participants were more exploratory than non-depressives in their decision-making. Depressive symptoms appear to influence basic mechanisms supporting choice behavior by reducing use of rich task representations and hindering performance during exploratory decision-making.

Keywords: Cognitive modeling; Depression; Dynamic decision making; Exploration vs. exploitation; Reinforcement learning.

Figure 1

The Leapfrog task: example choices over 100 trials. On any trial the lower option might, with a probability of 0.075, increase its reward by 20 points, surpassing the other option. The relative superiority of the two options alternates as their reward values “leapfrog” over one another. The lines represent the true reward values, the dots a participant's choices.

Figure 2

Performance and exploration rates. (A) The proportion of trials that the higher payoff option was chosen. (B) The proportion of trials that an exploratory choice was made. Depressives explored more often than non-depressives. (C) The proportion of exploratory choices for explore-optimal and exploit-optimal trials. Trials were classified based on the prescription of the Ideal Actor given the history of rewards. On exploit-optimal trials, depressives explored more often than non-depressives. Error bars reflect standard errors.

Figure 3

The proportion of participants best fit by each model. Depressives were more often best fit by the Naïve RL model than non-depressives.

Figure 4

Exploration rates as a function of number of successive exploitive choices. (A) Predictions from the two models. (B) Behavioral results for participants best-fit by the Ideal Actor. (C) Results for participants best fit by the Naïve RL model. Error bars reflect standard errors.