Asynchronous decoding of dexterous finger movements using M1 neurons

Abstract

Previous efforts in brain-machine interfaces (BMI) have looked at decoding movement intent or hand and arm trajectory, but current cortical control strategies have not focused on the decoding of dexterous [corrected] actions such as finger movements. The present work demonstrates the asynchronous decoding (i.e., where cues indicating the onset of movement are not known) of individual and combined finger movements. Single-unit activities were recorded sequentially from a population of neurons in the M1 hand area of trained rhesus monkeys during flexion and extension movements of each finger and the wrist. Nonlinear filters were designed to detect the onset of movement and decode the movement type from randomly selected neuronal ensembles (assembled from individually recorded single-unit activities). Average asynchronous decoding accuracies as high as 99.8%, 96.2%, and 90.5%, were achieved for individuated finger and wrist movements with three monkeys. Average decoding accuracy was still 92.5% when combined movements of two fingers were included. These results demonstrate that it is possible to asynchronously decode dexterous finger movements from a neuronal ensemble with high accuracy. This work takes an important step towards the development of a BMI for direct neural control of a state-of-the-art, multifingered hand prosthesis.

Fig. 1

Histograms of spike counts from sample neurons in monkey K show that the neuronal population contains both broadly-tuned neurons active during multiple finger movements (e.g K11404, K13905), and highly-tuned neurons active during only specific finger movements (e.g K13409, K13506). The histograms (blue) represent spiking activity during the 100 ms movement period directly preceding switch closure, while the scatter points (green) represent background spiking activity during non-movement periods.

Fig. 2

A) The timecourse of neuronal activity from monkey K shows a gradual evolution of spiking activity in and around the time of switch closure (1 s). B) Although a clear increase in activity is evident, the effect varies across neurons and movement types (shown for 12 sample neurons from monkey K and 4 movement types), which adds to the complexity of the problem at hand.

Fig. 3

The timecourse of a sample neuron in monkey K during movement of individual fingers shows that the neuronal firing rate gradually increased and reached a peak at (or slightly before) the time of switch closure. A) Rather than training the Gating Classifier at only the time of switch closure, a B) “soft” trapezoidal membership function was used to train the classifier over a broad time period. The sensitivity of the Gating Classifier was improved by applying fuzzy decision boundaries and thresholding the output of the classifier. C) To train the Movement Classifier, however, only the neuronal firing rate during the 100ms period of activity directly preceding switch closure was used. This helped ensure that the Movement Classifier only captured neuronal activity associated with a specific finger movement, and not gross movements.

Fig. 4

A) The raw output from a single gating network shows that the predicted output (red) tracked the actual output (blue) very closely. Therefore, the gating classifier did a good job of decoding movement intent. B) The raw output from a single movement network shows that the predicted output neuron activity (red) was specific to the movement type to be decoded (blue). Therefore, the movement classifier did a good job of decoding the movement type.

Fig. 5

A–C) shows the final decoded output for 3 different classifiers where only a single gating and movement network were used (shown for monkey K using 20 neurons for decoding). Each classifier fared poorly individually, but when D) a committee network is employed the overall final decoded output showed a marked improvement.

Fig. 6

Asynchronous decoding results for individuated and combined finger movements. For individuated movements, decoding accuracy was as high as 99.8% ± 0.1% for monkey K using 40 neurons, and 95.4% ± 1.0% using only 25 neurons. Although lower, decoding accuracy was still 96.2% ± 1.8% for monkey C and 90.5% ± 2.1% for monkey G using 40 neurons. When combined movements were included, average decoding accuracy was 92.5% ± 1.1% for all 18 movement types using 40 neurons for monkey K.

Fig. 7

Real-time decoding results for each movement type. Ten out of 12 individuated movement types are decoded with >95% accuracy for monkey C, and all 12 individuated movement types are decoded with >99% accuracy for monkey K. When combined movement movements are included, 13 out 18 movement types are decoded with >99% for monkey K.