Stimulating human prefrontal cortex increases reward learning

Abstract

Work in computational psychiatry suggests that mood disorders may stem from aberrant reinforcement learning processes. Specifically, it has been proposed that depressed individuals believe that negative events are more informative than positive events, resulting in higher learning rates from negative outcomes (Pulcu and Browning, 2019). In this proof-of-concept study, we investigated whether transcranial direct current stimulation (tDCS) applied to dorsolateral prefrontal cortex, as commonly used in depression treatment trials, might change learning rates for affective outcomes. Healthy adults completed an established reinforcement learning task (Pulcu and Browning, 2017) in which the information content of reward and loss outcomes was manipulated by varying the volatility of stimulus-outcome associations. Learning rates on the tasks were quantified using computational models. Stimulation over dorsolateral prefrontal cortex (DLPFC) but not motor cortex (M1) increased learning rates specifically for reward outcomes. The effects of prefrontal tDCS were cognitive state-dependent: tDCS applied during task performance increased learning rates for wins; tDCS applied before task performance decreased both win and loss learning rates. A replication study confirmed the key finding that tDCS to DLPFC during task performance increased learning rates specifically for rewards. Taken together, these findings demonstrate the potential of tDCS for modulating computational parameters of reinforcement learning that are relevant to mood disorders.

Keywords: Affective bias; Brain stimulation; Learning rate; Prefrontal tDCS; Reinforcement learning.

Fig. 1

(A) Schematic representation of a trial on the Information Bias Learning Task (IBLT; Pulcu and Browning, 2017). After showing the fixation cross and total amount of money won, two abstract shapes are presented on either side of the cross. Once the participant has chosen one of the shapes via a button press, a black frame appears around that shape and a win and loss outcome appear successively in randomised order. A win outcome leads to an increase of 10p, whereas a loss outcome represents a decrease of 10p from the total amount of money won. The total amount of money is updated at the start of the next trial. The aim of the task is to maximise earnings by learning the probabilities of the win and the loss appearing over the respective shapes. (B) The four possible outcomes on a task trial. The win and the loss outcomes are independently associated with one of the shapes, allowing for a shape to be associated with one, both, or neither of the outcomes at a given time. (C) Volatility of the win (green) and loss (red) outcomes across blocks of the Information Bias Learning Task (IBLT). Volatility for the two outcomes is manipulated independently across task blocks, either switching between 20% and 80% choice-outcome associations or remaining stable at 50% choice-outcome associations. If both wins and losses are volatile, participants should rapidly update their predictions for both types of outcomes (i.e., have a high learning rate). In the ‘Win-volatile’ blocks participants should adopt a high learning rate for wins and a low learning rate for losses, whereas the opposite approach should be taken in the ‘Loss-volatile’ blocks. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

(A) Structure and timeline of the tDCS paradigm combined with tDCS applied during task performance (Study 1, 3, and 4). Participants completed six blocks of the Information Bias Learning Task (IBLT). The task started and ended with a ‘Both-volatile’ block in which win and loss outcomes are equally informative. The participants were then presented with two ‘Win-volatile’ and two ‘Loss-volatile’ blocks, with block type order counterbalanced across participants. tDCS was applied during Blocks 2–3 of the IBLT. (B) Study 2 structure and timeline, in which DLPFC tDCS was applied prior to task performance. (C) Simulation of the electric field (top) and normal component (bottom) induced in the brain by the bilateral prefrontal tDCS montage, with the anodal electrode (red) over the left DLPFC (F3) and the cathode (blue) over the right DLPFC (F4). (D) Simulation of the electric field induced in the brain by the bilateral motor cortex tDCS montage, with the anodal electrode (red) over left M1 and the cathode (blue) over right M1. While the electric field strength induced in the two hemispheres is similar, the left hemisphere primarily receives anodal (red), and the right hemisphere cathodal stimulation (blue)(normal component). (C) and (D) are adapted from (Laakso et al., 2016) with permission. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Prefrontal tDCS during task performance selectively increased win learning rates. (A) Prefrontal stimulation during task performance selectively increased learning rates for win outcomes in blocks where losses were volatile (*p < 0.05). (B) The valence-specific effect shown in A) was also cognitive-state specific, as prefrontal stimulation applied before task performance decreased both win and loss learning rates (*p < 0.05). (C) The valence-specific effect shown in A) was also anatomically specific, as stimulation over motor cortex (M1) had no such effect. Violin-plots show the distribution of learning rates by tDCS condition (sham = blue, active = orange). Dots and error bars represent the mean and SEM values. Jittered dots show participants’ individual data points averaged across task blocks. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Replication study. Planned comparisons in the replication study (Study 4) revealed that prefrontal tDCS during task performance induced a marginally significant increase in win learning rates, but only during the stimulation period (. = 0.05), replicating the key finding of Study 1 (Fig. 3). Violin-plots show the distribution of learning rates by outcome (wins = blue, losses = orange). Dots and error bars represent the mean and SEM values. Jittered dots show participants’ individual data points averaged across task blocks.