Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Abstract

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

Fig. 1. Striatal tTIS during reinforcement learning of motor skills in the MRI machine.

a, Motor learning task. The participants were required to squeeze a hand-grip force sensor (depicted in the upper right corner of the figure) to track a moving target (the larger circle with a cross in the centre) with a cursor (the smaller black circle)^,. Pre- and post-training assessments were performed with full visual feedback of the cursor and no reinforcement. In Reinf_ON and Reinf_OFF trials, the participants practised the task with or without reinforcement feedback, respectively. In Reinf_ON trials, the colour of the target varied in real time as a function of the participants’ tracking performance. b, Experimental procedure. The participants performed the task in the MRI machine with concomitant tTIS. Blocks of training were composed of 36 trials (4 pre-training, 24 training and 8 post-training trials) interspersed with short resting periods (represented as plus signs in the figure). The six training types resulted from the combination of three tTIS_TYPES and two Reinf_TYPES. c, Concept of tTIS. On the left, two pairs of electrodes are shown on a head model, and currents I₁ and I₂ are applied with frequencies f₁ and f₁ + Δf. On the right, the interference of the two electric fields within the brain is represented for two different locations with high and low envelope modulation. E₁(t) and E₂(t) represent the modulation of the fields’ magnitude over time. tTIS was delivered with a Δf of 20 or 80 Hz or as a sham (a ramp-up and immediate ramp-down of high-frequency currents with a flat envelope). d, Electric field modelling with the striatal montage. The colours show the temporal interference exposure (electric field modulation magnitude). e, Temporal interference exposure in the striatum and in the overlying cortex. The violin plots show the tTIS exposure distribution over the voxels in the striatum and cortex underneath the stimulation electrodes. The magnitude of the field in the cortex was extracted from the BNA regions underneath the stimulation electrodes (F3–F4 and TP7–TP8). The black bar represents the mean. Voxels with outlying tTIS exposure (±5 s.d. around the mean) were removed from the plot (21 values from a total of 46,479 considered voxels).

Fig. 2. Behavioural results.

a, Motor performance across training. The raw Error data (expressed in percentage of maximum voluntary contraction (MVC)) from the 24 participants are presented in the left panel for the different experimental conditions in bins of four trials. The increase in Error during training is related to the visual uncertainty (that is, the intermittent disappearance of the cursor) that was applied to enhance reinforcement effects. The three plots on the right represent the pre-training normalized Error in the tTIS_Sham, tTIS_20Hz and tTIS_80Hz blocks. Reinforcement-related benefits represent the improvement in the Error measured in the Reinf_ON and Reinf_OFF blocks during training (reflecting benefits in motor performance) or at post-training (reflecting benefits in learning). b, Averaged learning across conditions. The violin plot shows the Error distribution at post-training (expressed in percentage of pre-training) averaged across conditions, as well as individual participant data. A single-sample two-sided t-test showed that the post-training Error was lower than the pre-training level, indicating significant learning in the task (P = 0.013; n = 24 participants). c, Motor learning. The averaged Error at post-training (normalized to pre-training) and the corresponding individual data points in the different experimental conditions are shown in the left and right panels, respectively, for the participants included in the analysis (that is, after outlier detection; remaining n = 23). The reduction of Error at post-training reflects true improvement at tracking the target in test conditions (in the absence of reinforcement, visual uncertainty or tTIS). The LMM run on these data revealed a specific effect of tTIS_80Hz on reinforcement-related benefits in learning (analysis of variance (ANOVA) with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment). Learning was disrupted with Reinf_ON in the tTIS_80Hz condition compared with the tTIS_20Hz (P = 0.039) and tTIS_Sham (P < 0.001) conditions. d, Motor performance. The averaged Error during training (normalized to pre-training) and the corresponding individual data points in the different experimental conditions are shown in the left and right panels, respectively, for the participants included in the analysis (that is, after outlier detection; n = 23). The Error change during training reflects the joint contribution of the experimental manipulations (visual uncertainty, potential reinforcement and tTIS) to motor performance. The LMM run on these data showed a frequency-dependent effect of tTIS on motor performance, irrespective of reinforcement (ANOVA with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment). Motor performance was disrupted irrespective of reinforcement in the tTIS_20Hz (versus tTIS_Sham: P < 0.001) and tTIS_80Hz (versus tTIS_Sham: P < 0.001; versus tTIS_20Hz: P = 0.031) conditions. The data are represented as mean ± s.e.

Fig. 3. Striatal activity.

a, Striatal BOLD responses. A 3D reconstruction of the striatal masks used in the current experiment is surrounded by plots showing averaged BOLD activity for each mask in the different experimental conditions. An LMM run on these data showed higher striatal responses in the Reinf_ON than in the Reinf_OFF condition, but no effect of tTIS_TYPE and no interaction (n = 24 participants). The data are represented as mean ± s.e. b, Whole-brain activity associated with the behavioural effect of tTIS_80Hz on reinforcement motor learning. The correlation between tTIS-related modulation of striatal activity (tTIS_80Hz–tTIS_20Hz) and learning abilities in the Reinf_ON condition (n = 24) is shown. Significant clusters of correlation were found in the left putamen and bilateral caudate (t-contrast; uncorrected P = 0.001 at the voxel level; corrected cluster-based false discovery rate, P = 0.05). The lower panel shows individual robust linear regressions for the three significant regions highlighted in the whole-brain analysis.

Fig. 4. Striatum-to-frontal-cortex effective connectivity.

a, Motor network. A 3D reconstruction of the masks used for the motor network (that is, dorso-lateral putamen (dlPu), dorsal caudate (dCa), M1 and SMA) is shown on the left. The plot on the right shows the effective connectivity from motor striatum to motor cortex in the different experimental conditions (n = 24 participants). Note the increase of connectivity with tTIS_80Hz in the presence of reinforcement (tTIS_80Hz–Reinf_ON: P = 0.001 (versus tTIS_80Hz–Reinf_OFF) and P < 0.001 (versus tTIS_Sham–Reinf_ON)). b, Reward network. A 3D reconstruction of the masks used for the reward network (that is, ventro-medial putamen (vmPu), NAc, vmPFC and ACC) is shown on the left. The plot on the right shows the effective connectivity from motor striatum to motor cortex in the different experimental conditions (n = 24). ROIs were defined on the basis of the BNA. In a and b, the outputs of LMMs were analysed using ANOVA with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment. The data are represented as mean ± s.e.