Decoding Native Cortical Representations for Flexion and Extension at Upper Limb Joints Using Electrocorticography

Abstract

Brain-machine interface (BMI) researchers have traditionally focused on modeling endpoint reaching tasks to provide the control of neurally driven prosthetic arms. Most previous research has focused on achieving an endpoint control through a Cartesian-coordinate-centered approach. However, a joint-centered approach could potentially be used to intuitively control a wide range of limb movements. We systematically investigated the feasibility of discriminating between flexion and extension of different upper limb joints using electrocorticography(ECoG) recordings from sensorimotor cortex. Four subjects implanted with macro-ECoG (10-mm spacing), high-density ECoG (5-mm spacing), and/or micro-ECoG arrays (0.9-mm spacing and 4 mm × 4 mm coverage), performed randomly cued flexions or extensions of the fingers, wrist, or elbow contralateral to the implanted hemisphere. We trained a linear model to classify six movements using averaged high-gamma power (70-110 Hz) modulations at different latencies with respect to movement onset, and within a time interval restricted to flexion or extension at each joint. Offline decoding models for each subject classified these movements with accuracies of 62%-83%. Our results suggest that the widespread ECoG coverage of sensorimotor cortex could allow a whole limb BMI to sample native cortical representations in order to control flexion and extension at multiple joints.

Figure. 1.

Brain map reconstructions for patients P1, P2, P3, and P4 (A, B, C, and D, respectively). Blue rectangles surround the ECoG electrodes that were selected for analyses in each patient. For patient P3, two micro-ECoG arrays (circled in blue) located on the sensorimotor cortex were also used for analyses. The central sulcus is outlined in black. Post-experimental analysis showed that patient P1 displayed significant ictal activity at electrode sites marked by beige-colored shapes. Therefore, during common average referencing, ECoG signals from electrodes within the green dotted box were spatially referenced separately from the other electrodes within the solid blue box. Electrodes are marked with colors on the yellow-red spectrum to represent their contribution to classifying the 6 movements, where the decoding weights of electrodes are normalized to a scale from 0 to 1. Smaller (yellow) electrodes contributed little to decoding, while large (dark red) electrodes contributed most to decoding.

Figure. 2.

Plots showing accuracy of classifying 6 movements as a function of time, for patients P1, P2, P3, and P4 (A, B, C, and D, respectively). The blue trace represents average time-dependent classification accuracy after 10-fold cross validation, and the black horizontal trace represents average chance accuracy (1,000 shuffles) with respect to movement onset (black vertical line). The blue shading represents the standard deviation of the classification accuracies across the 10 folds. The red shaded regions represent where the decoding accuracy was significantly higher than chance (p < 0.05, FDR-corrected).

Figure. 3.

Confusion matrices are shown for patients P1, P2, P3, and P4 (A, B, C, and D, respectively). Classification accuracies were computed within a window of −0.12 to 0.9 s, −0.22 to 0.65 s, −0.18 to 1.15 s,and −0.54 to 1.25 s relative to movement onset for patients P1-P4, respectively. Abbreviations F Flx, F Ext, W Flx, W Ext, E Flx, and E Ext represent finger, wrist, and elbow flexion and extension. The labels on the vertical axis represent the visually cued (actual) movements, and the labels on the horizontal axis represent the movements predicted by the classifier. Dark blue to light green and yellow to dark red displays low to middle to high classification accuracy (0–1). Confusion matrices for joint classification are also shown for patients P1-P4 (E), in which flexion and extension trials for each joint were grouped together. Finally, accuracies for flexion vs. extension at each joint separately are included in a table (F).

Figure. 4.

Log-transformed HG power changes were only analyzed for the electrodes outlined by red boxes in A, B, E, and F for patients P1-P4. The central sulcus is outlined in black. C, D, G, and H display brain maps showing the movement-specific average of log-transformed HG power changes within the respective bounded time windows in those electrodes. The color and size of the electrodes were varied to illustrate significant log-transformed power changes (relative to baseline), with units of log10(V²/Hz). Green, blue, and yellow circles highlight the subset of electrodes that showed significantly different HG activity between flexion and extension of fingers, wrist, and elbow, respectively. Percentages of significantly active electrodes showing a significant difference between flexion and extension about a specific joint are displayed between the flexion and extension brain maps. Cyan triangles mark those electrodes that showed higher gamma power for flexion or extension of each joint. Beige shapes surrounding electrodes on the brain map of patient P1 (A) represent electrodes with high ictal and interictal activity.

Figure. 5.

A and B show the decoding contributions of the electrodes in PMIC and AMIC, respectively, used to classify the six movements. Electrodes are marked with colors on the yellow-red spectrum to represent their contribution to movement classification. Smaller (yellow) electrodes contributed little to decoding, while large (dark red) electrodes contributed most to decoding. Classification accuracy from PMIC electrodes was computed from a window between −0.144 s to 1.15 s relative to movement onset, and the accuracy from AMIC electrodes was computed from a window between 0.056 s to 1.15 s post-movement onset. The central sulcus is outlined in black. C and D show the confusion matrices resulting from classification models using PMIC and AMIC arrays, respectively. Abbreviations F Flx, F Ext, W Flx, W Ext, E Flx, and E Ext represent fingers, wrist, and elbow flexion and extension.