Intention estimation in brain-machine interfaces

Abstract

Objective: The objective of this work was to quantitatively investigate the mechanisms underlying the performance gains of the recently reported 'recalibrated feedback intention-trained Kalman Filter' (ReFIT-KF).

Approach: This was accomplished by designing variants of the ReFIT-KF algorithm and evaluating training and online data to understand the neural basis of this improvement. We focused on assessing the contribution of two training set innovations of the ReFIT-KF algorithm: intention estimation and the two-stage training paradigm.

Main results: Within the two-stage training paradigm, we found that intention estimation independently increased target acquisition rates by 37% and 59%, respectively, across two monkeys implanted with multiunit intracortical arrays. Intention estimation improved performance by enhancing the tuning properties and the mutual information between the kinematic and neural training data. Furthermore, intention estimation led to fewer shifts in channel tuning between the training set and online control, suggesting that less adaptation was required during online control. Retraining the decoder with online BMI training data also reduced shifts in tuning, suggesting a benefit of training a decoder in the same behavioral context; however, retraining also led to slower online decode velocities. Finally, we demonstrated that one- and two-stage training paradigms performed comparably when intention estimation is applied.

Significance: These findings highlight the utility of intention estimation in reducing the need for adaptive strategies and improving the online performance of BMIs, helping to guide future BMI design decisions.

Figure 1

Schematic of the two-stage training paradigms. In the first step of training, the monkey performs a 2D center out and back reaching task with the native arm. [i] A first pass BMI model is built using the recorded neural data and the kinematic data from the arm movements. In the second step of training, the monkey performs the same 2D center out and back reaching task, while using the first pass BMI model (KF). A second set of training data is collected during this online run to build the final BMI model. This model-build is carried out [iia] without training set modifications (Re-KF) and [iib] with intention estimation modifications (IEMs) of the training set (ReFIT-KF). The bolded training paradigm indicates the method of training used in the ReFIT-KF algorithm [48]. The blue boxes schematize the subject’s online cursor control activity; the black boxes schematize the training data, which is derived from the online cursor control.

Figure 2

Effect of intention estimation modifications on the performance of three decoders. (a) The average distance from the current BMI cursor location to the target for center-out reaches. The thick line indicates the dial-in-time. Data from Monkey L, 2011-07-21 and 2011-07-22. (b) Bar graph of the average acquire time required to successfully obtain the target for Monkey L. The white line indicates the average time the subject reaches the target. (c) Same as in (a) but for Monkey J, 2012-07-26, 2012-07-27. (d) Same as in (b) but in Monkey J.

Figure 3

Effects of intention estimation on tuning characteristics. (a) Tuning curve characteristics using training data from two representative channels with (black) and without (grey) intention estimation for channel 41, Monkey J, 2011-06-10. Error bars denote standard error. (b) Same as (a) but for channel 76, Monkey J, 2011-06-10. (c) Histogram of the ratio of the tuning curve modulation when intention estimation modifications are applied over not applied across all viable channels for Monkey J (blue, averaged data across 60 days) and Monkey L (red, averaged data across 55 days). (d) Histogram of the ratio of per-channel firing rate variance when intention estimation is applied over not applied across all viable channels for Monkey J (blue, averaged data across 60 days) and Monkey L (red, averaged data across 55 days).

Figure 4

Tuning shifts between training sets and respective online testing sets for each component of the two-stage training paradigm. (a) Histograms of the shifts in PD between the native hand training data and the online first pass decoder, KF (stage i); first-pass decoder training set and the online final decoder data, Re-KF (stage iia); the first-pass decoder training set with intention estimations modifications (IEMs) and the online final decoder data, ReFIT-KF (stage iib). Red bars indicate statistically significant deviations, and the green line indicates the bootstrapped distribution of shifts due to noise. (b) Shifts in PD per channel ordered from the most (channel 1) to least contributory channels (channel 85) for stage i, iia, and iib. The channels that have statistically significant shifts in PD are colored in red. Data from Monkey J, 2012-07-26.

Figure 5

Tuning shifts between training sets and respective online testing sets, as in Fig 4. Data from Monkey L, 2011-07-21.

Figure 6

Schematic of the one-stage training paradigms. A final BMI control model is built from the native hand control training data without [ia] and with intention estimation modifications (IEMs) [ib]

Figure 7

Comparison of the online performance of one and two-stage decoders with intention estimation. (a) The average distance to target is plotted for center-out reaches using hand control (green), KF (blue), ReFIT-KF (yellow), and FIT-KF (red). Data from Monkey L, aggregated across 6 experimental days. (b) The average acquire times to successfully hold a target are plotted for the four control modalities, where the white line indicates the first time the subject reaches the target without necessarily holding it in Monkey L (c) Same comparison as (a) but in Monkey J, data aggregated across 7 experimental days. (d) Same as (b) but in Monkey J.

Figure 8

Tuning shifts between training and online testing sets for a one-stage training paradigm. (a) Histograms of the shifts in PD between the hand training data versus the online first pass decoder, KF (stage ia), and between the hand training data with intention estimation modifications versus the online first pass decoder, FIT (stage ib). (b) Shifts in PD ordered from most to least contributory channels. Data from Monkey L, 2011-04-21.

Figure 9

Tuning shifts between training sets and respective online testing sets, as in Fig 8. Data from Monkey J, 2011-05-19.