Sequential exploration in the Iowa gambling task: Validation of a new computational model in a large dataset of young and old healthy participants

Abstract

The Iowa Gambling Task (IGT) is one of the most common paradigms used to assess decision-making and executive functioning in neurological and psychiatric disorders. Several reinforcement-learning (RL) models were recently proposed to refine the qualitative and quantitative inferences that can be made about these processes based on IGT data. Yet, these models do not account for the complex exploratory patterns which characterize participants' behavior in the task. Using a dataset of more than 500 subjects, we demonstrate the existence of sequential exploration in the IGT and we describe a new computational architecture disentangling exploitation, random exploration and sequential exploration in this large population of participants. The new Value plus Sequential Exploration (VSE) architecture provided a better fit than previous models. Parameter recovery, model recovery and simulation analyses confirmed the superiority of the VSE scheme. Furthermore, using the VSE model, we confirmed the existence of a significant reduction in directed exploration across lifespan in the IGT, as previously reported with other paradigms. Finally, we provide a user-friendly toolbox enabling researchers to easily and flexibly fit computational models on the IGT data, hence promoting reanalysis of the numerous datasets acquired in various populations of patients and contributing to the development of computational psychiatry.

Fig 1. Directed exploration in the IOWA Gambling Task.

(A) In the IGT, participants must sample 4 decks of card associated with gains and losses whose magnitudes vary in a probabilistic manner. Unbeknownst to participants, decks C and D are advantageous despite offering smaller gains, because the losses are respectively very low or rarely encountered. The columns below each deck report the empirical minimum, maximum and average magnitude of gains (all decks have a 100% gain probability), the frequency, maximum and mean magnitude of losses, as well as the overall expected value. (B) Here, we propose a new computational model accounting for trial-by-trial choices in the IGT. The “Value and Sequential Exploration” (VSE) model consist of two learning modules tracking respectively the net amount of money generated (exploitation weights, top) and the exploration weights of each deck (dependent upon the time elapsed since the last selection of that deck). These two weights are then summed and transformed into a probability of choosing each deck through a classical softmax. As such, the model implements a straightforward arbitration between reward- and information-seeking drives. (C) The VSE architecture was justified by the discovery of a peculiar choice pattern in the IGT, that it can reproduce for some combination of parameters. Namely, the probability of choosing 4 different decks within 4 consecutive trials (a pattern referred to as the SE index) was largely above chance levels, especially in the beginning of the task.

Fig 2. Model comparison.

(A) Model comparison treating model attribution as a fixed effect showed that the VSE model outperformed all other models, independently of the penalization for complexity implemented by the different estimators (BIC, AIC or Free Energy). The least difference, observed with the VPP model using Free Energy, still reflected decisive evidence in favor of the VSE model (Bayes Factor > 100)[19]. (B) Bayesian model comparison treating model attribution as a random effect also showed that the VSE model outperformed all other models on the 3 estimators (exceedance probability superior to >0.99 in every case). (C) The VSE model was also the best model for predicting single decision, both for fitted and simulated choice data. For fitted choice data, accuracy refers to choice probabilities as produced by the best-fitting parameters. For simulated choice data, accuracy refers to choice probabilities as produced by the best-fitting parameters based on simulated data (in both cases, accuracy equals 1 if the actual choice corresponds to the highest probability under the model, 0 otherwise).

Fig 3. Ability to account for sequential exploration and recovery rates across models.

(A) Compared to other models, the VSE model predicted a higher number of sequential exploration (SE) events. (B) The VSE model had also the highest sensitivity to sequential exploration, as shown by a d-prime analysis. (C) The results of the model recovery analysis clearly showed that the EV, VSE and PVL models could be efficiently recovered while the PVL-delta and VPP models showed the worst performance, with the ORL model having an intermediate status. Note that the red areas correspond to situation where a model which was not used to generate the data (indicated by the dashed vertical line) produced better fit than the generator model (see Methods for details). (D) The linear regression curves linking the parameters initially estimated based on participants’ choices and those estimated based on simulated choices showed that the VSE model had the highest parameter recovery performance (i.e minimal deviation from the identity diagonal).

Fig 4. Model-based analysis of IGT behavior in a healthy aging cohort.

(A) Using the VSE model as a tool to refine the characterization of age-related changes in risky decision-making, we observed that old and young healthy adults differed on 3 parameters. Relative to young adults, old adults had a lower decay parameter (reflecting a faster forgetting of exploitation weights), a higher consistency (reflecting a more deterministic choice policy) and a lower exploration bonus (reflecting a lower tendency to engage sequential exploration). (B) The age-related reduction in the exploration bonus was reflected into a significantly lower SE index in old as compared to young participants.