Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia

Abstract

Computer-mediated connections between human motor cortical neurons and assistive devices promise to improve or restore lost function in people with paralysis. Recently, a pilot clinical study of an intracortical neural interface system demonstrated that a tetraplegic human was able to obtain continuous two-dimensional control of a computer cursor using neural activity recorded from his motor cortex. This control, however, was not sufficiently accurate for reliable use in many common computer control tasks. Here, we studied several central design choices for such a system including the kinematic representation for cursor movement, the decoding method that translates neuronal ensemble spiking activity into a control signal and the cursor control task used during training for optimizing the parameters of the decoding method. In two tetraplegic participants, we found that controlling a cursor's velocity resulted in more accurate closed-loop control than controlling its position directly and that cursor velocity control was achieved more rapidly than position control. Control quality was further improved over conventional linear filters by using a probabilistic method, the Kalman filter, to decode human motor cortical activity. Performance assessment based on standard metrics used for the evaluation of a wide range of pointing devices demonstrated significantly improved cursor control with velocity rather than position decoding.

Figure 1

Training paradigm. (a) Illustrations of a series of open-loop (OL) training (left), closed-loop (CL) training (middle) and testing blocks (right). In the OL block (1–1.5 min period of recording), the training cursor (TC) was moved by a technician or a computer program to reach a target. After one to four OL blocks, CL training blocks included a feedback cursor (FC), controlled by decoded neural signals in real time, that was also displayed on the monitor to provide visual feedback to the participants. In testing blocks, the participants volitionally controlled a neural cursor (NC) to reach and dwell (for 500 ms) on targets. (b) An example of the computer-generated speed profile for the TC. Here, a speed value of 0.1 s⁻¹ represents moving 10% of the screen height per second. Across all the sessions included in this study, the speed profile duration varied from 1.5 s to 4.5 s. (c) An example of the TC positions collected for 8 min during the random pursuit-tracking task used for training position-based decoding algorithm (from the session S3-40). The screen space was normalized to [−0.6, 0.6] in the horizontal axis (denoted X here) and [−0.5, 0.5] in the vertical axis (Y).

Figure 2

Illustration of the NC path performance measures. (a) MDC counts the number of movement direction changes parallel to the task axis (for instance, MDC = 2 in this example). (b) ODC counts the number of direction changes orthogonal to the task axis (ODC = 2). (c)–(d) ME measures movement deviation by computing the average of the absolute deviations |y_i| from the task axis, and MV measures the standard deviation of y_i . Two cases are illustrated in which both ME and MV are large (c) or ME is large but MV is small (d). (Illustration large derived from MacKenzie et al [28].)

Figure 3

Analysis of kinematic tuning during training. (a) Histograms showing the percentage of units with optimal lags ranging from 0 to 2 s (1126 neuronal units from 17 recording sessions in two participants (S3 and A1)). (b) The correlation coefficients (CC) between individual firing rates and kinematic parameters—position (white) versus velocity (gray)—during open-loop training blocks. The median of the CC values is represented by bars and 25% and 75% percentiles by vertical lines. (c) The percentage of units significantly correlated (methods) with either position or velocity (p < 0.01; t-test). The number of units (N) recorded in each session is denoted in parentheses. (d) The percentage of units that showed stronger (and significant) correlation with one of position or velocity compared to the other.

Figure 4

Decoding performance improvement with training. Evolution of the average CC between the TC and the FC trajectories of position or velocity during closed-loop (CL) training blocks for S3 (a)and A1(b). CC = 1 represents perfect correlation between the TC and the FC. The CC was measured for the two groups of CL blocks separated based on the decoding method used: the position-based linear filter (white) and the velocity-based Kalman filter (gray). The bars to the left side of a vertical dashed line illustrate the chance level for decoding accuracy (methods) for each decoding method. The bars and the vertical lines to the right side represent the average CC and the 95% confidence interval during each of the 1st to the 4th CL blocks.

Figure 5

Consistent directional tuning between training and testing. Sample spike trains are shown for three example units in one session (S3-261), in which S3 performed the center-out-back task during training (left) and testing (right). Black tics in the raster plots show the times when neuronal spikes occurred after target onset. Spike raster plots are arrayed according to the direction of movement (four directions during training and eight during testing). The arrows indicate the preferred directions estimated for each unit. The bars below the spike raster plots show the histogram of spikes for 1 s in 100 ms non-overlapping windows.

Figure 6

Changes in directional tuning between training and testing. The spike trains of two sample units from A1-197 (a) and A1-224 (b) are shown. The left and right columns illustrate tuning during training and testing, respectively. The arrows indicate the estimated preferred direction (PD).

Figure 7

Statistics of directional tuning change. (a) The statistics of tuning across training and testing is illustrated in terms of the percentage of subsets of units with particular properties in S3 (over six sessions) and in A1 (over three sessions). The neural population was divided into four subsets including: units showing no directional tuning across the session, units directionally tuned only during training, units tuned only during testing, and units tuned during both training and testing. The last subset is further divided into smaller groups including: one showing significant PD changes and the other showing no PD change. (b) For units that changed PD, the bars indicate the percentage of units that changed by the indicated number of degrees (0–30, 30–60, etc) for S3 (blue) and A1 (red).

Figure 8

The NC movement paths decoded by the position-based linear filter in S3. The NC movement paths from the center to four radial targets performed by S3 using the position-based linear filter in three recording sessions (S3-40, S3-48 and S3-54). (Top) 80 NC paths (20 per target) denoted by black lines are shown per session. The yellow squares approximately represent target positions and sizes. N denotes the number of recorded units. (Bottom) the mean NC path toward each target is shown for each session. Different line colors denote the mean path for different targets: blue: 0°; gray: 90°; green: 180°; black: 270°. To obtain the mean path, all the NC paths per target were linearly interpolated to make each path (with different duration) have the same number of sample points. The mean path per target was then computed over these interpolated paths. Note that some mean paths did not reach the target. This illustrates that for some trials, the NC failed to reach the target before timeout expired.

Figure 9

The NC movement paths decoded by the velocity-based Kalman filter in S3. The NC movement paths from the center to four radial targets performed by S3 using the velocity-based Kalman filter in six recording sessions (S3-254, S3-261, S3-280, S3-282, S3-285 and S3-287). 44–80 NC paths (10–20 per target) are shown together with the mean NC paths. N denotes the number of recorded units.

Figure 10

The NC movement paths decoded by the position-based linear filter for A1. The NC movement paths performed by A1 using the position-based linear filter in two recording sessions (A1-85 and A1-91). (Top) 40 NC paths (10 per target) are shown. (Bottom) The mean NC path toward each target is shown for each session.

Figure 11

The NC movement paths decoded by the velocity-based Kalman filter for A1. The NC movement paths performed by A1 using the velocity-based Kalman filter in three recording sessions (A1-197, A1-216 and A1-224). (Top) 12–43 NC paths (2–11 per target) are shown. (Bottom) The mean NC path toward each target is shown for each session.

Figure 12

The distribution of neural cursor speed. (a) The NC speed distributions with position-based linear filter decoding (blue) or velocity-based Kalman filter decoding (red) are presented. The NC speed data were collected from nine sessions of S3 with approximately 40 000 time samples for each of position and velocity decoding. (b) Mean cursor speed is shown as a function of 2D cursor location for position-based decoding. Each pixel depicts the mean speed of the NC in that region of the screen. The highest cursor speeds are found at the periphery of the movement space near the targets. (c) Mean cursor speed for velocity-based decoding is shown; the highest speeds are found in the center of the screen with lower speeds near the targets. For both (b)and(c), the dotted squares represent the target size and location.

Figure 13

The NC paths decoded by the velocity-based Kalman filter and linear filter. The NC paths performed by S3 using the velocity-based decoders during the eight-target center-out task are shown. Performance results from three recording sessions (S3-408, S3-412 and S3-418) are shown where both the linear filter and the Kalman filter were sequentially tested. The individual NC paths and the mean path to each target made using either (a) the linear filter or (b) the Kalman filter are shown. Eight targets were located at (0°,45°,90°, 135°, 180°, 225°, 270° and 315°). N denotes the number of units used to build the decoding filter.