Learning to control a brain-machine interface for reaching and grasping by primates

Abstract

Reaching and grasping in primates depend on the coordination of neural activity in large frontoparietal ensembles. Here we demonstrate that primates can learn to reach and grasp virtual objects by controlling a robot arm through a closed-loop brain-machine interface (BMIc) that uses multiple mathematical models to extract several motor parameters (i.e., hand position, velocity, gripping force, and the EMGs of multiple arm muscles) from the electrical activity of frontoparietal neuronal ensembles. As single neurons typically contribute to the encoding of several motor parameters, we observed that high BMIc accuracy required recording from large neuronal ensembles. Continuous BMIc operation by monkeys led to significant improvements in both model predictions and behavioral performance. Using visual feedback, monkeys succeeded in producing robot reach-and-grasp movements even when their arms did not move. Learning to operate the BMIc was paralleled by functional reorganization in multiple cortical areas, suggesting that the dynamic properties of the BMIc were incorporated into motor and sensory cortical representations.

Figure 1. Experimental Setup, Behavioral Tasks, Changes in Performance with Training, EMG Records during Pole and Brain Control, and Stability of Model Predictions

(A) Behavioral setup and control loops, consisting of the data acquisition system, the computer running multiple linear models in real time, the robot arm equipped with a gripper, and the visual display. The pole was equipped with a gripping force transducer. Robot position was translated into cursor position on the screen, and feedback of the gripping force was provided by changing the cursor size. (B) Schematics of three behavioral tasks. In task 1, the monkey's goal was to move the cursor to a visual target (green) that appeared at random locations on the screen. In task 2, the pole was stationary, and the monkey had to grasp a virtual object by developing a particular gripping force instructed by two red circles displayed on the screen. Task 3 was a combination of tasks 1 and 2. The monkey had to move the cursor to the target and then develop a gripping force necessary to grasp a virtual object. (C–E) Behavioral performance for two monkeys in tasks 1–3. The percentage of correctly completed trials increased, while the time to conclude a trial decreased with training. This was true for both pole (blue) and brain (red) control. Horizontal (green) lines indicate chance performance obtained from the random walk model. The introduction of the robot arm into the BMIc control loop resulted in a drop in behavioral performance. In approximately seven training sessions, the animal's behavioral performance gradually returned to the initial values. This effect took place during both pole and brain control. (F) Stability of model predictions of hand velocity during long pole-control sessions (more than 50 min) for two monkeys performing task 1. The first 10 min of performance were used to train the model, and then its coefficients were frozen. Model predictions remained highly accurate for tens of minutes. (G) Surface EMGs of arm muscles recorded in task 1 for pole control (left) and brain control without arm movements (right). Top plots show the X-coordinate of the cursor; plots below display EMGs of wrist flexors, wrist extensors, and biceps. EMG modulations were absent in brain control.

Figure 2. Performance of Linear Models in Predicting Multiple Parameters of Arm Movements, Gripping Force, and EMG from the Activity Frontoparietal Neuronal Ensembles Recorded in Pole Control

(A) Motor parameters (blue) and their prediction using linear models (red). From top to bottom: Hand position (HPx, HPy) and velocity (HVx, HVy) during execution of task 1 and gripping force (GF) during execution of tasks 2 and 1. (B) EMGs (blue) recorded in task 1 and their prediction (red). (C) Contribution of neurons from the same ensemble to predictions of hand position (top), velocity (middle), and gripping force (bottom). Contributions were measured as correlation coefficients (R) between the recorded motor parameters and their values predicted by the linear model. The color bar at the bottom indicates cortical areas where the neurons were located. Each neuron contributed to prediction of multiple parameters of movements, and each area contained information about all parameters. (D–F) Contribution of different cortical areas to model predictions of hand position, velocity (task 1), and gripping force (task 2). For each area, ND curves represent the average prediction accuracy (R²) as a function of number of neurons needed to attain it. Contributions of each cortical area vary for different parameters. Typically, more than 30 randomly sampled neurons were required for an acceptable level of prediction. (G–I) Comparison of the contribution of single units (blue) and multiunits (red) to predictions of hand position, velocity, and gripping force. Single units and multiunits were taken from all cortical areas. Single units' contribution exceeded that of multiunits by approximately 20%.

Figure 3. Long-Term Functional Changes in Multiple Cortical Areas

(A) Color-coded (red shows high values; blue, low values) representation of individual contributions measured as the correlation coefficient (R) of neurons to linear model predictions of hand position for 42 training sessions. The average contribution steadily increased with training. The bar on the left indicates cortical location of the neurons. (B–E) Average contribution of neurons located in different cortical areas (PMd, M1, S1, and SMA, respectively) to hand position prediction during 42 recording sessions. (F) Average contribution for the whole ensemble to hand position prediction versus hand velocity predictions. A linear increase in contribution was observed only for predictions of hand position.

Figure 4. Directional Tuning in Frontoparietal Ensemble during Different Modes of Operation in Task 1

(A–D) Directional tuning during pole control (A), brain control with arm movements (tuning assessed from cursor movements) (B), brain control without arm movements (tuning assessed from cursor movements) (C), and brain control with arm movements (tuning assessed from pole movements) (D). In each of the color-coded diagrams (red shows high values and blue low values; see color scale), the rows depict normalized directional tuning for individual cells. Because of the high directional tuning values of some cells (e.g., that shown in [H]), a color scale limit was set at 0.3 to allow color representation of the largest possible number of cells. Each tuning curve contains eight points that have been interpolated for visual clarity. Correspondence of tuning patterns under different conditions has been quantified using correlation coefficients (shown near lines connecting the diagrams). The highest correspondence was between tuning during pole control and brain control with arm movements. A much less similar pattern of direction tuning emerged during brain control without arm movements. Polar plots near each diagram show average directional tuning for the whole neural ensemble recorded. They indicate an average decrease in tuning strength and shifts in the preferred direction of tuning as the operation mode was switched from pole to brain control. Spread of preferred directions (90° corresponds to uniformly random distribution) is indicated near each polar plot. (E–G) Scatterplots comparing DTD (maximum minus minimum values of tuning curves) during pole control versus brain control with and without arm movements. DTD during brain control was consistently lower than during pole control. This effect was particularly evident during brain control without arm movements. (H–J) Changes in directional tuning for individual neurons under different conditions. Blue shows pole control; red, brain control with arm movements (tuning assessed from pole movements); and green, brain control without arm movements. The first illustrated cell (H) was tuned only when the monkey moved its arm, more so during pole control. The second cell (I) had similar tuning during all modes of operation, but tuning depth was the highest for pole control and the lowest for brain control without arm movements. The third cell (J) was better tuned during brain control.

Figure 5. Plasticity of Directional Tuning during Training in Brain Control without Arm Movements

Conventions are as in Figure 4. (A) Directional tuning profiles during four sessions in pole control (task 1). Percentages of correctly performed trials are indicated for each session. (B) Scatterplots comparing directional tuning during pole versus brain control for the same sessions. For each day, DTD was on average higher in pole control. (C) Directional tuning during brain control for the same sessions as in (A). Note the emergence of a population pattern in which a group of neurons (with some exceptions) exhibits a similar preferred direction. This is manifested by a decrease in the spread of preferred directions (shown near polar plots). Notice also a gradual rotation of the population preferred direction (see polar plots) with training. (D) Gradual changes in DTE during one representative session of brain control without arm movements. This 60-min session was split into 5-min periods, five of which are shown. (E) Improvement in behavioral performance during a single session (same as in [D]) . (F) Decrease in the spread of preferred directions during that session. (G) Increase in average tuning depth during the same session.

Figure 6. Ensemble Encoding of Gripping Force, Plasticity of Directional Tuning, and Neuronal Contribution to Model Performance during Learning to Control the BMIc for Reaching and Grasping

(A) Perievent time histograms (PETHs) in task 2 for the neuronal population sampled in monkey 1. The plots on top are color-coded (red shows high values; blue, low values). Each horizontal row represents a PETH for a single-neuron or multiunit activity. PETHs have been normalized by subtracting the mean and then dividing by the standard deviation. PETHs are aligned on the gripping force onset (crossing a threshold). Plots at the bottom show the corresponding average traces of gripping force. Note the general similarity of PETHs in pole (left) and brain (right) control in this relatively easy task. Cortical location of neurons is indicated by the bar on the top left. Note the distinct pattern of activation for different areas. (B) Changes in the mean contribution of neurons from different cortical areas to model predictions during training of monkey 1 in task 2. (C) Increases in directional tuning for six cortical areas during training in pole control in task 3. (D and E) Increases in neuronal contribution to linear models predicting hand position (blue), hand velocity (red), and gripping force (black) during learning task 3 in both monkeys. (F and G) Representative robot trajectories and gripping force profiles in an advanced stage of training in task 3 during both pole and brain control. The bottom graphs show trajectories and the amount of the gripping force developed during grasping each virtual object. The dotted vertical lines in the panels indicate the end of reach phase and the beginning of grasp phase. Note that during both modes of BMIc operation, the patterns of reaching and grasping movements (displacement followed by force increase) were preserved.