Optimal feedback control successfully explains changes in neural modulations during experiments with brain-machine interfaces

Abstract

Recent experiments with brain-machine-interfaces (BMIs) indicate that the extent of neural modulations increased abruptly upon starting to operate the interface, and especially after the monkey stopped moving its hand. In contrast, neural modulations that are correlated with the kinematics of the movement remained relatively unchanged. Here we demonstrate that similar changes are produced by simulated neurons that encode the relevant signals generated by an optimal feedback controller during simulated BMI experiments. The optimal feedback controller relies on state estimation that integrates both visual and proprioceptive feedback with prior estimations from an internal model. The processing required for optimal state estimation and control were conducted in the state-space, and neural recording was simulated by modeling two populations of neurons that encode either only the estimated state or also the control signal. Spike counts were generated as realizations of doubly stochastic Poisson processes with linear tuning curves. The model successfully reconstructs the main features of the kinematics and neural activity during regular reaching movements. Most importantly, the activity of the simulated neurons successfully reproduces the observed changes in neural modulations upon switching to brain control. Further theoretical analysis and simulations indicate that increasing the process noise during normal reaching movement results in similar changes in neural modulations. Thus, we conclude that the observed changes in neural modulations during BMI experiments can be attributed to increasing process noise associated with the imperfect BMI filter, and, more directly, to the resulting increase in the variance of the encoded signals associated with state estimation and the required control signal.

Keywords: brain-machine interfaces; computational motor control; neural modulations; optimal feedback control; process noise.

Figure 1

Schematic model of movement control during BMI experiments under the hypothesis that the brain implements OFC (A), and detailed block diagram of neural activity generation (B). (A) The brain model receives noisy proprioceptive (y_P) and visual (y_C) measurements from the hand and cursor, corrupted by proprioceptive and visual measurement noise, ω_P and ω_V, respectively. These noisy measurements are integrated with prior predictions from the internal model to generate optimal state estimates $\widehat{x}$_k|k and control signal u_k, which are encoded by the neural activity. The control signal is corrupted by hand process noise ξ_u. The BMI filter is trained based on the neural activity in pole control and then used to move the cursor in brain control. (B) The cumulative bin rate, Γ(k), at time step k, is modulated by the encoded signals $S={[s_1,\cdots ,s_M]}^T$ including the estimated state $\widehat{x}$_k|k and control signals u_k. The spike-count N(k) is generated as a doubly stochastic Point process given the rate parameter Γ(k). Here we consider the special case of linear encoding, where Γ(k) is a linear combination of the encoded signals (including the estimated speed and the magnitude of the control signal), and doubly stochastic Poisson processes (DSPP), where the spike count N(k) has a Poisson distribution with rate Γ(k).

Figure 2

BMI filter performance as a function of the number of neurons and the process noise factor. Results are based on 15 simulated sessions of 20 min pole control, each with a different, randomly selected targets, but all with the same set of neural tuning weights. In each session the BMI filter was trained on the last 10 min and its performance was quantified by the coefficient of correlation between the actual and predicted velocity in the first 10 min. Each data point and error bar (in B,C) depict the average and standard deviation of the coefficient of correlations across the 15 sessions for a specific process noise factor k_u and total number of neurons N_n. The effect of k_u is evaluated with N_n = 50 (B), while the effect of N_n is evaluated with k_u = 0.1 (C). A fixed 1:1 ratio is kept between the M1-like and PMd-like simulated neurons.

Figure 3

Movement trajectories. Representative traces of the cursor during simulated pole control, brain control and noisy pole control (with α_rec = 0.035). To facilitate comparison, the same set of targets (marked by green circles) and desired reaching times was used in these three simulations.

Figure 4

Cross correlation between velocity and neural activity during pole control. Averaged cross correlation of: (A) recorded data from PMd units (B) recorded data from M1 units (C) simulated PMd-like neurons (D) simulated M1-like neurons. Error bars depict standard deviations across non-overlapping 2-min intervals. Open circles and stars indicate cross-correlations that are significantly lower than the peak at 0 s in (A,C), and at −0.2 s at (B,D) with significance level of either p = 0.05 (standard) or p = 0.005 (Bonferroni corrected for 10 multiple comparisons), respectively.

Figure 5

Estimated POM and PKM of simulated neural activity. Estimated POM and PKM were computed by first estimating the POM and PKM of the binned spike-counts of individual neurons in 2-min intervals, and then averaging across the relevant neurons and across the 10 intervals of each control mode. Mean estimated POM (A) and PKM (B) across all neurons and across neurons that encode only the estimated state (PMd-like) or also the control signals (M1-like). Error bars depict the standard deviations across the 10 intervals. Scatter plot of estimated POM (C) and PKM (D) in brain vs. pole control for 78 out of 100 sessions (each with a different set of tuning weights) in which the performance of the BMI filter satisfied 0.65 < R($\widehat{v}$, v) < 0.85. Dashed lines depict the identity relationship.

Figure 6

Estimated POM and PKM during simulated noisy pole control. Noisy pole control includes both the baseline process noise due to the control signal, and an additional process noise, simulating the contribution of the BMI filter reconstruction error, as quantified by the magnitude of α_rec in Equation (13). At each α_rec and for each adaptation option (updated or not), the average estimated POM and PKM were computed from a single 20 min session of simulated pole control as detailed in the caption of Figure 5. Standard deviations were computed across the 10 2-min intervals and stars indicate the estimated POM (for not-updated case) that are significantly larger than the estimated POM at the standard process noise (α_rec = 0) as determined by Wilcoxon rank sum test, with Bonferroni corrected significance level p = 0.005, corrected for 10 multiple comparisons. All sessions were performed with the same randomly selected targets and neural tuning weights.

Figure 7

Sensitivity of estimated POM to proprioceptive measurement noise (A) and magnitude of baseline process noise (B). The proprioceptive measurement noise factor k_m quantifies the ratio between the variance of the proprioceptive and the visual measurement noise (same factor for both position and velocity measurements). The process noise factor k_u quantifies the magnitude of hand process noise, as specified in Table 1. Each sensitivity analysis was conducted by simulating 15 sessions with the same set of neurons but different sets of targets. The mean estimated POM over the simulated neurons was computed for each session and the average and standard deviations across the 15 sessions are plotted.

Figure 8

Effect of internal model variations during pole control. The value of the mass (A) or the coefficient of friction (B) in the internal model is normalized to the value of the actual parameter. Simulations are conducted with either the baseline proprioceptive measurement noise characteristic of pole control or infinite proprioceptive measurement noise characteristic of BCWOHM. At each parameter value, and for each level of the proprioceptive measurement noise, the average estimated POM and PKM were computed from a single 20 min session of simulated pole control as detailed in the caption of Figure 5. Standard deviations were computed across the 10 2-min intervals and stars indicate estimated POMs that are significantly larger than the estimated POM at the nominal parameter (internal model factor = 1), as determined by Wilcoxon rank sum test with Bonferroni corrected significance level p = 0.007, corrected for 7 multiple comparisons. All sessions were performed with the same randomly selected targets and neural tuning weights.