A brain-computer interface that evokes tactile sensations improves robotic arm control

Abstract

Prosthetic arms controlled by a brain-computer interface can enable people with tetraplegia to perform functional movements. However, vision provides limited feedback because information about grasping objects is best relayed through tactile feedback. We supplemented vision with tactile percepts evoked using a bidirectional brain-computer interface that records neural activity from the motor cortex and generates tactile sensations through intracortical microstimulation of the somatosensory cortex. This enabled a person with tetraplegia to substantially improve performance with a robotic limb; trial times on a clinical upper-limb assessment were reduced by half, from a median time of 20.9 to 10.2 seconds. Faster times were primarily due to less time spent attempting to grasp objects, revealing that mimicking known biological control principles results in task performance that is closer to able-bodied human abilities.

Fig. 1:

Overview of the bidirectional BCI system. (A) The participant used the intracortical BCI to control a robotic prosthesis in real time, controlling all five dimensions (dark blue arrows) continuously from the start to the end of the trial. (B) Four microelectrode arrays were implanted in the left hemisphere. Arrays in motor cortex (blue) recorded neural activity to control the prosthesis. Arrays in somatosensory cortex (red) delivered stimulation pulses, evoking sensory percepts referred to the hand. (C) Torque measurements from the robotic hand controlled stimulation of individual electrodes. Colored grids (adapted from Flesher et al. 2016) represent electrodes and locations on the hand where stimulation evoked a percept. Index finger torque was used to drive stimulation of the index finger sensation and middle finger torque was used to drive stimulation of electrodes associated with the middle, ring, and pinky finger. (D) Stimulation current amplitude was modulated by torque using a linear transformation. (E) Example raster plot of neural data recorded from motor cortex and decoded into endpoint velocities using an optimal linear estimator. (F) Overhead view of the Action Research Arm Test (ARAT). Different objects (not all shown) were positioned at the presentation location (green dot), grasped, and then placed on the platform (green box) as quickly as possible. (G) Overhead view of the object transfer task showing the object (gray), transit (red) and target (green) zones. (Image Credit: Kenzie Green)

Fig. 2:

Effect of ICMS on ARAT performance. (A) ARAT scores when ICMS feedback was provided were significantly better than prior ARAT scores (*p = 0.005), which occasionally employed ICMS feedback (blue dots), and to data from the current experiment without ICMS feedback (*p = 0.029). Red lines indicate median scores. (B) Histogram of successful trial times completed with (blue) and without (gray) ICMS tactile feedback. Median trial times (dotted lines) were significantly faster with ICMS (*p < 0.0001). Hatched bars represent trials completed in under five seconds. (C) Empirical cumulative distribution of individual trial times, including failed trials, shown on a log-normalized axis. Vertical red lines indicate when 50% of successful trials were completed. Data to the left of the vertical green line represent trials completed in under five seconds. Shading indicates the 95% confidence bounds, calculated with Greenwood’s formula. (D) Amount of time spent in each phase of the ARAT task. Red lines are medians, box outlines are interquartile ranges, and whiskers are the range of the data excluding outliers (red ‘+’). All task phases were faster when ICMS feedback was provided (*p<0.001, **p<0.0001, Table S2). (A-D) Significance was assessed with a Wilcoxon rank-sum test.

Fig. 3:

Effect of ICMS on object transfer performance. (A) Amount of time spent in each task zone, per transfer, by feedback condition (n = 20 trials per feedback condition). Data for all trials are shown with the mean value indicated by the red lines and the whiskers indicating one standard deviation. The amount of time spent in the object and target zones decreased significantly with ICMS feedback (*p = 0.002 and 0.048, t-test, respectively). (B) Distribution of average path lengths in the object zone per trial for the two feedback conditions, computed as the total path length divided by the number of transfers. Mean path length decreased with ICMS feedback (*p = 0.0007, t-test). (C) Spatial map of the average amount of time spent in each location in the workspace per transfer. Each individual square represents a 2 x 2 cm region of the workspace. The color indicates the average amount of time spent in each location per transfer. Without stimulation, more time was spent near the object in the object zone as shown by the darker red colors in the object zone. Red lines indicate zone boundaries.