Behavioral and neural correlates of visuomotor adaptation observed through a brain-computer interface in primary motor cortex

Abstract

Brain-computer interfaces (BCIs) provide a defined link between neural activity and devices, allowing a detailed study of the neural adaptive responses generating behavioral output. We trained monkeys to perform two-dimensional center-out movements of a computer cursor using a BCI. We then applied a perturbation by randomly selecting a subset of the recorded units and rotating their directional contributions to cursor movement by a consistent angle. Globally, this perturbation mimics a visuomotor transformation, and in the first part of this article we characterize the psychophysical indications of motor adaptation and compare them with known results from adaptation of natural reaching movements. Locally, however, only a subset of the neurons in the population actually contributes to error, allowing us to probe for signatures of neural adaptation that might be specific to the subset of neurons we perturbed. One compensation strategy would be to selectively adapt the subset of cells responsible for the error. An alternate strategy would be to globally adapt the entire population to correct the error. Using a recently developed mathematical technique that allows us to differentiate these two mechanisms, we found evidence of both strategies in the neural responses. The dominant strategy we observed was global, accounting for ∼86% of the total error reduction. The remaining 14% came from local changes in the tuning functions of the perturbed units. Interestingly, these local changes were specific to the details of the applied rotation: in particular, changes in the depth of tuning were only observed when the percentage of perturbed cells was small. These results imply that there may be constraints on the network's adaptive capabilities, at least for perturbations lasting only a few hundreds of trials.

Fig. 1.

Schematic of the applied perturbation, highlighting both the angular deviation and the speed reduction. A: schematic of activity during a movement in the control session. Thin gray lines represent the contributions of each neuron to the population vector, where each arrow points in the neuron's preferred direction (PD) and is scaled by its normalized firing rate. Thick black arrow represents the population vector. B: the same neural activity as represented in A, but decoded as if it occurred in the perturbation session. Half of the neurons are being decoded at a 90° angle to their original PDs. Thick black line shows the new population vector, with the thick gray line showing the control population vector, for reference. The perturbation shows both an angular deviation (θ) and a speed reduction (α). C: plot of the expected global rotation (θ) as a function of the percentage of cells rotated (p) when the decoding preferred directions (dPDs) are rotated by 90°. Circles denote experimental conditions we tested. D: speed gain (α) plotted as a function of the extent of the perturbation (ϕ) when 50% of the cells are rotated. Again, circles denote experimental conditions we tested (see Table 2 for a list of these experiments).

Fig. 2.

Trajectories show evidence of adaptation and predictive compensation. A: examples of single trials taken from the control, early perturbation (EP), and late perturbation (LP) sessions of 1 of the (50%, 60°, V) experiments (notation indicates 50% of the cells were rotated by 60° with no invisible zone). Each color denotes a different target. Dotted circles indicate the halfway point where the angular error is assessed. Dashed lines indicate the straight line between the origin and target, for reference. B: average trajectories from all 19 experiments of the (50%, 60°, V) condition. Format is the same as in A. C: average trajectories from the (50%, 60°, I) experimental condition (where 50% of the cells were rotated by 60° with an invisible zone), demonstrating the reduction in error between the EP and LP session trajectories. Dotted circle shows the location of the invisible zone. Dashed lines indicate trajectories from the EP session, and solid lines indicate trajectories from the LP session.

Fig. 3.

Speed (A) and standard deviation (SD; B) of the average trajectory in each session, plotted as a function of time elapsed since target presentation. Asterisks denote target acquisition time; trials end with the administration of reward. In B, SD = $\sqrt{\left({\text{SD}}_X^2+{\text{SD}}_Y^2\right)}$, where SD_X and SD_Y denote SDs in the X and Y components of position, respectively. C, control session; EW and LW, early and late washout sessions, respectively.

Fig. 4.

The overall pattern of angular errors is consistent across experimental conditions. Each plot gives the mean signed angular error as a function of the session within the experiment. Positive numbers denote errors in the direction of the applied rotation, negative numbers denote errors in the opposing direction, and vertical bars denote ±SE. The perturbation condition of each experiment is shown above its plot, and the number of experiments (n) performed in each condition is given at top right of each plot. Occasionally, the subjects would stop working before an adequate number of trials had been collected to compute the average error in the EW or LW session. Numbers in parentheses above the EW and LW bars state the number of experiments used to compute these values. Note the lack of error bars for the (50%, 30°, I) and (50%, 45°, I) LW cases, which only included data from 1 experiment each. Plots are arranged according to the extent of the rotational perturbation (Table 2).

Fig. 5.

Learning curves suggest multiple timescales of adaptation. A and B: to combine data across multiple experimental conditions, it is necessary to normalize by the effective perturbation. A: angular errors during the EP (circles) and EW sessions (diamonds), plotted as a function of the expected angular error (computed as described in methods). Open and filled symbols denote experiments with and without an invisible zone, respectively. B: same as A, but now the measured EP and EW errors have been normalized by the corresponding expected errors (as computed using Eq. 6). These data indicate that normalized errors may be compared across different experimental conditions. C: adaptation as a function of the number of successful trials. Left section shows data from the control session, middle section shows data from the perturbation session, and right section is from the washout session (section breaks are denoted with vertical dashed lines). Jagged gray line shows the normalized error as a function of the number of successful trials, averaged over all of the experiments. Black lines show the best biexponential (solid) and single-exponential (dotted) fits to this data, respectively. Inset shows a close up of the first 80 trials after the perturbation was applied. Note that not all experimental sessions contained the same number of successful trials. We show data only for trial numbers reached in at least 25 experiments.

Fig. 6.

The overall pattern of movement time changes is consistent across experimental conditions. Each plot shows the average time it took to move the cursor from the center of the workspace to the presented target as a function of the session within each experiment. Overall format is the same as for Fig. 4. Horizontal black lines denote the average time at which the cursor had moved one-half the distance to the targets, i.e., the average time at which the angular errors were assessed. These times were not significantly different between experiments with and without an invisible zone.

Fig. 7.

Adaptive speed increases are rapid but show limited dynamic range. A: cursor speed during the LP session plotted as a function of the cursor speed in the EP session. Roughly equal numbers of experiments show increases as show decreases between the 2 sessions. Data are from the 113 experiments that had a visuomotor gain reduction. Black line indicates the identity line. B: speed during the EP session plotted as a function of expected speed (calculated as the corresponding speed in the control session times the visuomotor gain reduction factor α). Speeds in the EP session are typically higher than expected. Format is the same as for A. C: response gain during the EP session for each applied visuomotor gain reduction. A value of 1 would indicate no adaptive response. The response gain required to fully compensate for the speed reduction is the inverse of the applied gain and is denoted by the horizontal black lines. Although the response gains for all experiments with applied gain reductions of <0.9 are statistically significant, the gains are much less than required to fully compensate for the speed reduction.

Fig. 8.

Firing rates of single neurons change in response to the perturbation. Data shown are from 1 example nonrotated neuron during a (50%, 60°, V) experiment. Center: average firing rate of the neuron plotted as a function of the target direction during the control (black), perturbation (red), and washout sessions(blue). Solid lines show the log-linear tuning curve fits, whereas data points and vertical lines show the means ± SE of the firing rates at each target direction. Surrounding plots show firing rate of the neuron to 8 of the 16 targets as a function of the number of sequential trials to that target. Target directions are indicated above each plot (as well as by the polar position of the plot). The zero value on each abscissa and the first vertical dotted line indicate when the perturbation was applied; the second vertical line indicates when the perturbation was removed.

Fig. 9.

Simulation showing 3 different compensation mechanisms that could all be responsible for an observed firing rate change in a rotated neuron. At right of each plot is an arrow schematic that shows the population response. Each thin arrow points in the decoding direction of 1 neuron (the direction in which it pushes the cursor) with its length proportional to the neuron's firing rate. Black arrows denote rotated neurons, gray arrows denote nonrotated neurons, and thick black dashed arrow indicates the population vector average (the direction of cursor movement). At left of each plot is a tuning curve of 1 of the rotated neurons showing the firing rate as a function of aiming direction. The dashed line shows this cell's tuning curve during the control session, where it had a PD of 0°, and the open circle indicates its firing rate when a target at 0° was presented in the control session. In each panel, the subject is trying to hit a target placed at 0° under the perturbation. Solid black lines and filled circles show the tuning curve and firing rate of the cell for the various compensation cases. A: no-compensation case. The subject is aiming at a target located at 0° (directly to the right), but the perturbation causes the cursor to move at 45°. B: re-aiming compensation. The subject aims toward −40° to create a movement nearer the target. C: re-weighting compensation. The subject aims directly at the target, and the rotated population contributes less to the overall cursor movement. D: re-mapping compensation. The subject aims at 0°, and the tuning curves of the rotated cells are altered to map this aiming direction to a decoded direction that is closer to the aiming direction.

Fig. 10.

Re-aiming points move to counter the applied perturbation. A: 2 examples of re-aiming point changes measured during the (50%, 60°, V) experimental condition (left, −60°; right, +60°). The blue end of each arc shows the location of 1 target's re-aiming point measured in the control session, whereas the red end shows its location measured in the perturbation session. The corresponding black crosses denote the 16 target locations. The arcs and crosses have been offset from the circle by random amounts to aid visibility. B: average rotation of the re-aiming points plotted as a function of the expected perturbation. The average rotation is computed as the mean angular shift of the re-aiming point (averaged across all 16 targets), where positive values are opposite to the applied perturbation (and therefore represent compensatory changes). Data from experiments with and without invisible zones are combined in the plot, since there were no visible differences between the 2 conditions.

Fig. 11.

Adaptation-related changes in tuning curve parameters, broken down by experimental condition and arranged from left to right according to the extent of the applied rotation. All changes are measured as the difference between the value in the perturbation session and the control session. Values above each plot are P values of a t-test comparison between parameter changes of the rotated and nonrotated neurons (ns indicates P > 0.05). No attempt was made to correct for multiple comparisons. A: changes in preferred direction (ΔPD). Positive values are in the direction of the applied perturbation. B: changes in modulation depth (ΔMD).

Fig. 12.

Re-weighting occurs only when the percentage of perturbed cells is small. A and B: histograms of the ΔMD (perturbation MD − control MD) for all units, shown separately for the 25% (A) and 50% experiments (B). C and D: histograms of the change in control ratio (perturbation − control) for the 25% (C) and 50% experiments (D).

Fig. 13.

Computing the error reduction that can be attributed to global and local mechanisms. Cursor movement direction can be estimated by decoding firing rates simulated with different re-aiming point and tuning curve combinations. Each solid black arrow indicates the direction of cursor movement that results from decoding these simulated firing rates. Data are from one of the (25%, 90°, V) experiments. A: firing rates are simulated with the tuning curves and re-aiming points measured in the control session and decoded with the control session decoder. The dashed arrows indicate the target directions, for reference. The average angular error between the cursor movement directions and the target directions is indicated. B: same calculation as in A, but now the firing rates are decoded with the perturbation session decoder. This indicates the error that would result if the subject did not adapt at all. C: firing rates are simulated with re-aiming points from the control session but with tuning curves from the perturbation session, decoded with the perturbation session decoder. This indicates the movements that would result from re-tuning compensation only. Red arrows indicate the “no-compensation” cursor movement directions, for reference. D: same as in C, but using re-aiming points from the perturbation session and tuning curves from the control session. E: firing rates are simulated using re-aiming points and tuning curves from the perturbation session.

Fig. 14.

Histograms showing the residual angular error predicted from different combinations of re-aiming points and tuning curves, as described in Fig. 13. nc, Error with no compensation; rt, remaining error after re-tuning only; ra, remaining error after re-aiming only; p, error in perturbation session with full compensation; c, error in control session. Each histogram shows data from a different perturbation condition, denoted at top left with the number of experiments (n) run in that perturbation condition.

Fig. 15.

Error reduction that can be attributed to global (re-aiming) or local (re-tuning) adaptation mechanisms. The error reduction is calculated as the difference between the error in the no-compensation case and the error with only re-aiming (ra) or only re-tuning (rt) allowed. The percentage of the total error accounted for by each method is given above each bar. Experiments are arranged according to the extent of the rotational perturbation (Table 2).

Fig. 16.

Dose-response effects. The error reduction that can be attributed to re-tuning and re-aiming is plotted as a function of the total error induced by the perturbation. Dotted lines show the linear regression fits of each relationship (re-aiming: y = 0.33 + 0.60x, P < 10⁻¹⁰, F-test; re-tuning: y = −0.24 + 0.12x, P = 0.003).