Real-time linear prediction of simultaneous and independent movements of two finger groups using an intracortical brain-machine interface

Abstract

Modern brain-machine interfaces can return function to people with paralysis, but current upper extremity brain-machine interfaces are unable to reproduce control of individuated finger movements. Here, for the first time, we present a real-time, high-speed, linear brain-machine interface in nonhuman primates that utilizes intracortical neural signals to bridge this gap. We created a non-prehensile task that systematically individuates two finger groups, the index finger and the middle-ring-small fingers combined. During online brain control, the ReFIT Kalman filter could predict individuated finger group movements with high performance. Next, training ridge regression decoders with individual movements was sufficient to predict untrained combined movements and vice versa. Finally, we compared the postural and movement tuning of finger-related cortical activity to find that individual cortical units simultaneously encode multiple behavioral dimensions. Our results suggest that linear decoders may be sufficient for brain-machine interfaces to execute high-dimensional tasks with the performance levels required for naturalistic neural prostheses.

Keywords: brain-machine interface; hand prosthesis; intracortical; linear decoder; multiple simultaneous targets; primary motor cortex.

Figure 1.. Experimental description.

(A) The monkey was seated in front of a screen displaying a virtual hand with his left hand placed in a manipulandum. Positions of the index and middle/ring/small (MRS) finger groups were measured by the manipulandum (right side of A) synchronously with the neural activity. The position measurements or the decoded finger positions were used to actuate the virtual hand, depending on the stage of the experiment. Targets were pseudo-randomly presented in a center-out pattern based on the postures in B or in a pattern where target positions were pseudo-randomly placed along each finger’s dimension, not separated by more than 50% of the range. (B) Two-dimensional space for visualizing the hand movements. Postures shown are at +30% compared to rest, which is at 50% between full flexion and full extension. I – index finger group, M – MRS finger group, F – flexion, E – extension, R – rest. (C) An example tuning curve from monkey N illustrating an SBP channel tuned to index extension and MRS flexion movements. Asterisks indicate significant difference from the average activity across the experiment (two-sided two-sample Kolmogorov-Smirnov test, p < 0.001, corrected for false discovery rate). (D) Photographs of monkey W’s and monkey N’s intracortical Utah microelectrode array implants. Both implants were in right hemisphere. * indicates arrays used in this study. A – anterior, L – lateral, CS – central sulcus.

Figure 2.. Two-finger closed-loop Kalman filter decodes using spiking band power (SBP).

(A, B) Example closed-loop prediction traces from monkeys N and W using the standard Kalman filter, respectively. Targets are represented by the dashed boxes, internally colored to indicate the targeted finger with a border color representing whether the trial was acquired successfully (green, red if not). “I” means the index finger group and “MRS” means the middle/ring/small finger group. The mean path efficiency of the trials displayed in each window is presented at the top right. (C) Statistics for all closed-loop two-finger Kalman filter trials for monkeys N (left) and W (right). The red lines indicate the means, which are numerically displayed above each set of data, along with standard deviation. The statistic for each trial is represented by one dot in each plot, split into columns per monkey. “Succ Rate” means the percentage of total trials that were successfully acquired in time and “Path Eff” means path efficiency.

Figure 3.. Two-finger closed-loop ReFIT Kalman filter decodes.

(A, B) Example closed-loop prediction traces from monkeys N and W, respectively, using the ReFIT Kalman filter. Targets are represented by the dashed boxes, internally colored to indicate the targeted finger. “I” means the index finger group and “MRS” means the middle/ring/small finger group. The mean path efficiency of the trials displayed is presented at the bottom right. (C) Statistics for all closed-loop two-finger ReFIT Kalman filter decodes for monkeys N (left) and W (right). The red lines indicate the means, which are numerically displayed above each set of data, along with standard deviation. The statistic for each trial is represented by one dot in each plot. “Succ Rate” means the percentage of total trials that were successfully acquired in time and “Path Eff” means path efficiency. (D) Statistics of each type of velocity reorientation for ReFIT training with monkey N (see STAR Methods). “N” means velocities for each finger were negated if not pointing to that finger’s target, “R” means the velocities were rotated in the two-dimensional finger space towards the target, and “B” means both reorientations were used by concatenating velocities modified by N and R and repeating the neural activity. Asterisks indicate significance (p < 0.01, two-tailed two-sample t-test).

Figure 4.. Analysis of changes in SBP channel tuning through different stages of online virtual hand control.

(A) Preferred directions for the three normalized SBP channels with the highest manipulandum-control magnitude on one day for each monkey. Solid arrows with square markers are the preferred direction and magnitude for each channel during manipulandum control (Manip.), dashed arrows with asterisk markers are the same during Kalman filter control (KF), and dotted arrows with circle markers are the same during ReFIT Kalman filter control (RF). (B, C) Changes in preferred direction (B) and magnitude (C) of the 5 highest-magnitude manipulandum control channels for each of the 6 days for each monkey.

Figure 5.. Predictability of the activity of each movement from the activities of other movements.

The top and bottom rows of axes represent monkeys N and W, respectively, and the left and right columns of axes represent SBP and sorted unit results, respectively. A cell in each set of axes is colored based on the predictor’s regression coefficient (horizontal axis) when predicting the activity of a movement (vertical axis). Asterisks indicate statistical significance (based on each coefficient’s t-score, p < 0.001).

Figure 6.. Offline ridge regression decoding of all SBP channels, trained on either individual or combined finger group movements.

The central trace with shaded region are the average predicted behavior from all trials of the indicated movement ± standard deviation, aligned in time by movement onset (vertical gray line). Individual movements (two left columns) were decoded using a regression trained on combined movements (right two columns), and vice versa for combined movements (“Split Decode” dashed traces). The “Full Decode” solid traces represent the average decode given the full dataset to train the regression, with cross-validation. Blue traces correspond to the index group and yellow traces correspond to the MRS group. The yellow or blue lines near the top of each plot indicate significant differences between the mean predicted positions based on a bootstrap analysis on the differences (>95% one-sided confidence interval).

Figure 7.. Comparison between postural and movement tunings of SBP during the random task with manipulandum control.

Dashed traces in all plots represent zero z-score. (A) Tuning curves for three channels from each monkey. Asterisks indicate significant differences in preferred direction determined via 1,000-iteration bootstrap, p < 0.01, corrected for false discovery rate. (B) Movement tuning overlaid on postural tuning. The color at each location, or posture, is the SBP activity for the channel above extracted from the target holding period (i.e. at zero velocity). The white line represents the preferred postural direction. The solid black trace represents that channel’s SBP activity for movements in each direction, smoothed across 10% of the trials. The black scale bar represents 0.5 z-score and the black line from center indicates the preferred movement direction. (C-E) Statistics of the plots in (B) for the 40 most impactful channels to a linear regression decoder. (C) Comparison between preferred movement and postural directions. Each line represents one channel. (D) Difference in angle between the preferred movement and postural directions. Each dot represents one channel. The width of the violin at each angle difference indicates smoothed relative density. n.s. means not significant, s. means significant with p < 0.01, corrected for false discovery rate. (E) Magnitude of difference between the maximum or minimum activity across all postures and rest. The numbers on top represent the median difference. Asterisks indicate statistical difference, p < 0.01, one-sided two-sample t-test. (F) Eight channels of z-scored electromyography across all postures during the hold periods.