A recurrent neural network for closed-loop intracortical brain-machine interface decoders

Abstract

Recurrent neural networks (RNNs) are useful tools for learning nonlinear relationships in time series data with complex temporal dependences. In this paper, we explore the ability of a simplified type of RNN, one with limited modifications to the internal weights called an echostate network (ESN), to effectively and continuously decode monkey reaches during a standard center-out reach task using a cortical brain-machine interface (BMI) in a closed loop. We demonstrate that the RNN, an ESN implementation termed a FORCE decoder (from first order reduced and controlled error learning), learns the task quickly and significantly outperforms the current state-of-the-art method, the velocity Kalman filter (VKF), using the measure of target acquire time. We also demonstrate that the FORCE decoder generalizes to a more difficult task by successfully operating the BMI in a randomized point-to-point task. The FORCE decoder is also robust as measured by the success rate over extended sessions. Finally, we show that decoded cursor dynamics are more like naturalistic hand movements than those of the VKF. Taken together, these results suggest that RNNs in general, and the FORCE decoder in particular, are powerful tools for BMI decoder applications.

Figure 1

Network schematic of the FORCE decoder with inputs and outputs. The FORCE decoder is a recurrent neural network with effective internal connections defined by the matrix gJ + W_FW_O^T, with g a global scale factor. The input to the decoder is the set of binned spikes s₁(t), …, s₉₆(t) from the multi-electrode array implanted in PMd and is multiplied by the weights in the matrix W_I. The network output is a decoded, normalized version of the monkey's arm position and velocity in the x and y directions, denoted p_x(t), p_y(t), v_x(t), v_y(t) for the normalized position and velocity signals, respectively (red traces on the right). The decoder is trained solely by modifying the weights W_O in red. The output is fed back to the network through weights W_F, allowing the decoded positions and velocities to modify the network dynamics. Thus, the network dynamics are a combination of the binned spikes, the activity resulting from the internal connectivity and the decoded position and velocity signals. The actual decode during BMI was a linear combination of position and integrated velocity signals rescaled to the physical dimensions of the workspace.

Figure 2

Representative ABA block trials for monkey J and monkey L. Representative days are shown for both monkeys. Shown are the target acquire times for each trial (ms). First, 200+ hand trials were collected as a baseline at the beginning of the session (acquire times shown in light blue and blue). The VKF decoder was run for 300+ trials (light green and green), followed by the FORCE decoder for 300+ trials (acquire times shown in light red and red) and finally the VKF decoder again for 300+ trials. The block average acquire times are shown as black lines across the trial blocks. Trials numbers without data points were used for either training data or for `burn' trials that allowed the monkey to habituate the new decoder.

Figure 3

Mean distance to target. Top—the distance (cm) of the arm to the target at each time point averaged across all trials and reach directions for both monkeys. The thicker line is the duration between the first time the monkey acquired the target and the final time the monkey acquired the target. Light blue and blue show monkey J's and L's native hand performance. Light red and red show the FORCE decoder's performance for monkeys J and L in BMI mode. Light green and green show the VKF's performance for monkeys J and L in BMI mode. Bottom—the average last acquire times for both monkeys under all three conditions. Average last acquire time measures how long on average it takes the monkey to complete the task.

Figure 4

Average acquire time histograms. Left—the histograms of average acquire time (ms), the time it took the monkey to complete a successful trial, across all trials for monkey J. All trials with acquire times greater than 2000 ms are shown at `2000+' ms. (Light blue/native arm, light red/FORCE decoder and light green/VKF). Right—the histograms of average acquire time across all trials for monkey L (blue/native arm, red/FORCE decoder and green/VKF).

Figure 5

Mean cursor speed. Left—the mean cursor speed (cm s⁻¹), the magnitude of the velocity of the cursor, for monkey J after target onset. Light blue shows monkey J's native hand speed. Light red shows the speed of the cursor with the FORCE decoder in BMI mode. Light green shows the mean cursor speed during operation of the VKF in BMI mode. Right: same as the left panel, but for monkey L (blue/hand, red/FORCE, green/VKF).

Figure 6

Generalization of the FORCE decoder. The decoder was trained on center-out 8 reaches and then the task was switched to the pinball task. The hit rate, which is the number of successful target acquisitions per minute, is shown for monkey J using his native arm (light blue) and in BMI mode with the FORCE decoder (light red). The monkey was able to sustain performance for over an hour on a more general task. Data for the VKF not shown because the monkey would not work with the VKF decoder long enough to take meaningful data.