Decoding and geometry of ten finger movements in human posterior parietal cortex and motor cortex

Abstract

Objective. Enable neural control of individual prosthetic fingers for participants with upper-limb paralysis.Approach. Two tetraplegic participants were each implanted with a 96-channel array in the left posterior parietal cortex (PPC). One of the participants was additionally implanted with a 96-channel array near the hand knob of the left motor cortex (MC). Across tens of sessions, we recorded neural activity while the participants attempted to move individual fingers of the right hand. Offline, we classified attempted finger movements from neural firing rates using linear discriminant analysis with cross-validation. The participants then used the neural classifier online to control individual fingers of a brain-machine interface (BMI). Finally, we characterized the neural representational geometry during individual finger movements of both hands.Main Results. The two participants achieved 86% and 92% online accuracy during BMI control of the contralateral fingers (chance = 17%). Offline, a linear decoder achieved ten-finger decoding accuracies of 70% and 66% using respective PPC recordings and 75% using MC recordings (chance = 10%). In MC and in one PPC array, a factorized code linked corresponding finger movements of the contralateral and ipsilateral hands.Significance. This is the first study to decode both contralateral and ipsilateral finger movements from PPC. Online BMI control of contralateral fingers exceeded that of previous finger BMIs. PPC and MC signals can be used to control individual prosthetic fingers, which may contribute to a hand restoration strategy for people with tetraplegia.

Keywords: brain-computer interface (BCI); factorized representations; finger decoding; hand movement; motor cortex (MC); posterior parietal cortex (PPC); representational geometry.

Figure 1.

Alternating-cues, instructed-delay finger press task. Trial structure. Each rectangle represents the computer monitor display at each phase. Two cue variants, text and spatial, were trial-interleaved. In the spatial variant, the location of the highlighted circle corresponded to the cued finger. Trials without a highlighted circle indicated a No-Go cue. In the text variant, a highlighted letter (for example, ‘M’ for the middle finger) cued each finger. In both variants, the finger cue disappeared before the movement phase (Go) to separate planning and execution periods. Phase durations are listed in supplementary table 1.

Figure 2.

Reaction-time finger-press task with randomized cue location. When a letter was cued by the red crosshair, the participant looked at the cue and immediately attempted to flex the corresponding digit of the right (contralateral) hand. We included a No-Go condition ‘X’, during which the participant looked at the target but did not move their fingers. Visual feedback indicated the decoded finger 1.5 s after cue presentation. To randomize the saccade location, cues were located on a grid (three rows, four columns) in a pseudorandom order. The red crosshair was jittered to minimize visual occlusion. Reproduced from [54]. CC BY 4.0.

Figure 3.

Text-cued finger movement task with instructed-delay. Trial structure. Text cues indicate the hand (‘R’ or ‘L’) and the finger (e.g. ‘m’ for middle finger). After a delay period, a cue-invariant Go-icon instructs movement execution.

Figure 4.

PPC single neurons discriminate between attempted finger movements. (a) Single-trial firing rates for an example NS-PPC neuron during attempted movements of different fingers. (top) Markers correspond to the firing rate during each trial. Gapped vertical lines to the right of markers indicate ± S.D., and each gap indicates the mean firing rate. (bottom) Firing rates during thumb (T) and index (I) presses were higher than the No-go (X) baseline. Vertical bars indicate bootstrap 95% confidence intervals (CI) of the effect size versus No-go baseline. Half-violin plots indicate bootstrap distributions. (b) Mean smoothed firing rates for each finger movement for two example NS-PPC neurons, which respectively modulated for thumb/index movements (left) and fingers versus No-Go (right). Shaded areas indicate 95% CI. (c) Percentage of NS-PPC neurons that discriminated between finger movements in each analysis window (q < 0.05, FDR-corrected for 466 neurons). Line (blue) indicates mean across sessions. Markers (gray) indicate individual sessions. (d) Complementary empirical cumulative distribution function visualizing the proportion of NS-PPC neurons with ANOVA effect sizes (η ²) above the corresponding x-axis value. Line colors indicate analysis epoch. Vertical lines (gray) indicate Cohen’s thresholds [59] for small (η ² = 0.01), medium (η ² = 0.06), and large (η ² = 0.14) effect sizes. (e) Overlap of NS-PPC neurons that modulated significantly (q < 0.05) with large effect sizes (η ² > 0.14) during movement preparation (plan) and movement execution (move).

Figure 5.

Offline classification of finger movement from population activity. (a) Cross-validated confusion matrix for classifying attempted finger movement from NS-PPC neural activity during the movement execution epoch. 86% accuracy, 480 trials over four sessions. (b) Learning curve showing cross-validated accuracy as a function of the training dataset size. About 40 trials (less than seven trials per finger) are needed to achieve 80% accuracy. Shaded area indicates 95% CI over folds/sessions. (c) Neuron-dropping curve showing cross-validated accuracy as a function of recorded neurons. Neurons were aggregated across sessions. About 70 neurons are needed to achieve 80% accuracy. Shaded area indicates 95% interval over subpopulation resamples. (d) Hyperparameter sweep showing cross-validated classification accuracy as a function of decode window size. Input features were the average firing rates in the window [200 ms, 200 ms + window size] after Go-cue. Window durations of about 350 ms are necessary to achieve 80% accuracy. Shaded area indicates 95% CI over folds/sessions. (e) Cross-validated classification accuracy across the trial duration (500 ms sliding window). Shaded area indicates 95% CI over folds/sessions.

Figure 6.

Online BMI classification of individual finger movements. (a) Confusion matrix for participant NS (PPC), right-hand finger presses. 86% accuracy ± S.D. 4% over ten sessions, 4016 total trials. Reproduced from [54]. CC BY 4.0. (b) Confusion matrix for participant JJ (PPC + MC), right-hand finger presses. 92% accuracy ± S.D. 3% over eight sessions, 1440 total trials.

Figure 7.

Offline classification of finger presses from both hands. (a) Mean firing rates for each finger movement for an example NS-PPC neuron, which increases its firing rate for thumb movements. Shaded areas indicate 95% confidence intervals (CI). (b) Same as (a) for a second example NS-PPC neuron, which increases it firing rate for index movements. (c) Cross-validated confusion matrix for classifying right- and left-hand finger movements from NS-PPC neural activity. 70% accuracy, 1000 trials over ten sessions. (d) Same as (c) using recordings from JJ-PPC. 66% accuracy, 200 trials over two sessions. (e) Same as (c) using recordings from JJ-MC. 75% accuracy, 200 trials over two sessions.

Figure 8.

Representational geometry of contralateral and ipsilateral finger movements. (a) Cross-validated squared Mahalanobis distances between NS-PPC activity patterns during the contralateral/ipsilateral finger press task. Distances were averaged over the ten sessions. (b) Non-matching (different finger-type, different hand) finger pairs have larger distances than matching (same finger-type, different hand) finger pairs. Each circle is one element of the dissimilarity matrix of an individual session, aggregated across ten sessions. (c) Example schematic of perfect factorization along hand and finger-type components. Line styles indicate groups of parallel, identical vectors. A factorized code generalizes linearly across each component axis. For example, the Rm population activity can be constructed from the summation: Li + left→right + index→middle. For visual clarity, figure only shows three finger-types (index, middle, ring). (d) Representational geometry of finger movements corresponding to NS-PPC distances (a), visualized in 2-D using MDS. We used Generalized Procrustes analysis (with scaling) to align across ten sessions. Ellipses show S.E. across sessions. Scale bars shown. Vectors with matching line-styles match each other, suggesting that the neural code is factorized. (e) Linear decoders generalized (supplementary figure 8) across finger-type to classify hand (left) and across hand to classify finger-type (right) (p < 0.001, permutation test), indicating that movement representations were factorized across finger-type and hand dimensions.