Smoking automaticity and tolerance moderate brain activation during explore-exploit behavior

Abstract

The adaptive trade-off between exploration and exploitation is a key component in models of reinforcement learning. Over the past decade, these models have been applied to the study of reward-seeking behavior. Drugs of addiction induce reward-seeking behavior and modify its underlying neurophysiological processes. These neurophysiological changes may underlie a behavioral shift from a flexible, exploratory mode to a focused, exploitative mode, which precedes the development of inflexible, habitual drug use. The goal of this study was to investigate the relationship between explore/exploit behavior and drug addiction by examining the neural correlates of this behavior in cigarette smokers. Participants (n=22) with a range of smoking behaviors completed a smoking dependence motives questionnaire and played a 6-armed bandit task while undergoing functional magnetic resonance imaging (fMRI). Exploratory behavior produced greater activation in the bilateral superior parietal and bilateral frontal cortices than exploitative behavior. Exploitative behavior produced greater activation in the bilateral superior and middle temporal gyri than exploratory behavior. fMRI data and orthogonalized smoking dependence motive scores were entered into multiple linear regression analyses. After controlling for nicotine tolerance, smoking automaticity positively correlated with activation in the same bilateral parietal regions preferentially activated by exploratory choices. These preliminary results link smoking dependence motives to variation in the neural processes that mediate exploratory decision making.

Keywords: Automaticity; Bandit task; Exploit; Explore; Neuroimaging; Smoking.

Figure 1

Example of payoff values per trial for each arm (i.e., slot machine) of the bandit task. Each colored line represents a different arm. Participants selected one arm to play each trial. Payoff values were determined by a biased random walk (Equation 1).

Figure 2

Statistical parametric maps for the selection phase of the bandit task showing regions of activation for Explore>Exploit (red) and Exploit>Explore (blue). AG: angular gyrus; BS: brainstem; CB: cerebellum; MFG: middle frontal gyrus; IPS: intraparietal sulcus, OCC: occipital cortex; PCG: precentral gyrus; PCL: paracingulate gyrus; PCN: precuneus; SPL: superior parietal lobule; STG & MTG: superior & middle temporal gyrus. Color bar represents T-scores, p < .005, cluster size 190 voxels.

Figure 3

Statistical parametric maps for the outcome phase of the bandit task showing regions where reward values were associated with increased (red) and decreased (blue) activation. AC: anterior cingulate; CN: caudate nucleus; PC: posterior cingulate; PCN: precuneus; SFG: superior frontal gyrus. Color bar represents T-scores, p < 0.005, cluster size 190 voxels.

Figure 4

Statistical parametric maps for the (a) selection and (b) outcome phases of the bandit task showing regions where WISDM component scores correlated with brain activation. Positive automaticity scores and negative tolerance scores correlated with exploratory activation in the bilateral parietal regions (a), and positive craving scores correlated with increasing reward values in the posterior cingulate (b). IPS: intraparietal sulcus; PC: posterior cingulate; PtCG: postcentral gyrus. Color bar represents T-scores, p < 0.005, cluster size 190 voxels.