Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys

Abstract

Brain-machine interfaces (BMIs) employ the electrical activity generated by cortical neurons directly for controlling external devices and have been conceived as a means for restoring human cognitive or sensory-motor functions. The dominant approach in BMI research has been to decode motor variables based on single-unit activity (SUA). Unfortunately, this approach suffers from poor long-term stability and daily recalibration is normally required to maintain reliable performance. A possible alternative is BMIs based on electrocorticograms (ECoGs), which measure population activity and may provide more durable and stable recording. However, the level of long-term stability that ECoG-based decoding can offer remains unclear. Here we propose a novel ECoG-based decoding paradigm and show that we have successfully decoded hand positions and arm joint angles during an asynchronous food-reaching task in monkeys when explicit cues prompting the onset of movement were not required. Performance using our ECoG-based decoder was comparable to existing SUA-based systems while evincing far superior stability and durability. In addition, the same decoder could be used for months without any drift in accuracy or recalibration. These results were achieved by incorporating the spatio-spectro-temporal integration of activity across multiple cortical areas to compensate for the lower fidelity of ECoG signals. These results show the feasibility of high-performance, chronic and versatile ECoG-based neuroprosthetic devices for real-life applications. This new method provides a stable platform for investigating cortical correlates for understanding motor control, sensory perception, and high-level cognitive processes.

Keywords: BMI; ECoG; arm; asynchronous; brain-machine interface; decoding; electrocorticography; long-term.

Figure 1

Experimental design and decoding performance to detect 3D hand positions. (A) Locations of the 32 electrodes in monkey A and the 64 electrodes in monkey K, which were identified by computed tomography (CT) and magnetic resonance imaging (MRI). Reference electrodes are shown as gray circles. (B) In the asynchronous food-reaching task, each monkey was trained to reach for food offered by the experimenter in 3D space without explicit cues. The body-centered coordinates for measuring 3D hand positions from the top-down viewpoint are shown. (C) Schematic diagram depicting the prediction of a motor parameter M(t) from simultaneously recorded ECoG signals. Examples of 1.1 s of raw ECoG signal from one electrode, the corresponding scalogram, down-sampled scalogram matrix, and normalized scalogram matrix are shown (bottom row).

Figure 2

Asynchronous decoding of 3D hand position by PLS regression. (A) Representative example of prediction of X-, Y-, and Z-positions of hand movements during a 5-min validation session. The average correlation coefficients (r) between the predicted (blue) and observed (red) trajectories for all positions are shown. (B) Determination of the optimal numbers of PLS components for the decoding models. R² values (green, mean ± SD) and PRESS (blue) for the decoding models with different numbers of PLS components in two monkeys. For each experiment, the number of PLS components for the optimal decoding model was determined as that with the minimal PRESS (red).

Figure 3

Long-term stability of decoding performance for same-day and cross-day predictions. (A) Correlation coefficients (r) of same-day prediction after implantation, shown with fitted 1-degree polynomials (lines) and error bounds that contain at least 50% of the predictions (shaded). (B) Upper panels: correlation coefficients of cross-day prediction with duration between the model construction and prediction (light-color symbols), shown with medians of N consecutive data points (N = 13, 10, and 12 for A-DL, A-DF, and K-C, respectively, see text) (lines) and lower/upper quartiles (shaded) versus medians of corresponding durations. Results of same-day predictions are also shown at duration = 0 days (dark-color symbols), with their medians shown as horizontal lines. Lower panels: p-values (pv) for comparisons of every N consecutive cross-day data points with N same-day predictions. The threshold of 0.01 is shown as a horizontal line.

Figure 4

Characteristics of the decoding models for 3D hand positions. (A) Spatial contributions of different electrodes, W_s(ch), for 3D hand positions. Median W_s(ch) across decoding models obtained from different experiments are shown (n = 13, 10, and 12 for A-DL, A-DF, and K-C, respectively). For each position, the electrodes with contributions significantly greater than their median (p < 0.01) are circled by thicker lines. (B) Spectral contributions of different frequency bins, W_f(freq), for 3D hand positions. For each position, the frequency bins with contributions significantly greater than their median (p < 0.01) are marked with asterisks. (C) Temporal contributions of different time lags, W_t(lag), for 3D hand positions. For each position, the time lags with contributions significantly greater than their median (p < 0.01) are marked with asterisks.

Figure 5

Decoding of 7-DOF arm motion. (A) One representative example of prediction of 7-DOF joint angles of arm orientations during a 5-min validation session are shown with the average correlation coefficients (r) between predicted (blue) and observed (red) trajectories. (B) Determination of the optimal numbers of PLS components for the decoding models (representation as in Figure 2B). (C) Spatial contributions of different electrodes for 7-DOF joint angles (representation as in Figure 4A).