Real-Time Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals from Paralysed Patients

Abstract

Neuroprosthetic arms might potentially restore motor functions for severely paralysed patients. Invasive measurements of cortical currents using electrocorticography have been widely used for neuroprosthetic control. Moreover, magnetoencephalography (MEG) exhibits characteristic brain signals similar to those of invasively measured signals. However, it remains unclear whether non-invasively measured signals convey enough motor information to control a neuroprosthetic hand, especially for severely paralysed patients whose sensorimotor cortex might be reorganized. We tested an MEG-based neuroprosthetic system to evaluate the accuracy of using cortical currents in the sensorimotor cortex of severely paralysed patients to control a prosthetic hand. The patients attempted to grasp with or open their paralysed hand while the slow components of MEG signals (slow movement fields; SMFs) were recorded. Even without actual movements, the SMFs of all patients indicated characteristic spatiotemporal patterns similar to actual movements, and the SMFs were successfully used to control a neuroprosthetic hand in a closed-loop condition. These results demonstrate that the slow components of MEG signals carry sufficient information to classify movement types. Successful control by paralysed patients suggests the feasibility of using an MEG-based neuroprosthetic hand to predict a patient's ability to control an invasive neuroprosthesis via the same signal sources as the non-invasive method.

Figure 1. Experimental paradigm and system overview.

(a) Experimental paradigm of the open-loop session. To begin, one of the movement types, grasp or open, was presented on the screen in front of the patient, followed by two “timing cues” and an “execution cue” at an interval of 1 s. The patient then attempted to move the affected hand as instructed at the timing of the “execution cue.” Each movement type was repeated four times. (b) System overview of the real-time prosthetic hand control. MEG signals from 84 parietal sensors, denoted by red dots, were acquired in real-time and analysed on a single computer. The prosthetic hand was controlled according to decoders that inferred the timing of movement intention and types of performed movements. The patient controlled the prosthetic hand by watching the screen representing the prosthetic hand and following the instructions for movements.

Figure 2. Measured magnetic fields.

(a) Normalized slow components of the MEG signals (SMF) of subject 1 are shown during the attempts to use the completely paralysed (affected) right hand and during actual movements of the intact left hand. The SMFs were acquired from 0 to 500 ms relative to the execution cue for each sensor and colour-coded according to the colour bar at the location of each sensor. Black arrows indicate the sensors used in plots (b,c). R, right; L, left. (b,c) Upper panels show averaged power spectra of the MEG signals recorded during attempted hand grasping by the paralysed hand (b) or actual hand grasping by the intact hand (c) of subject 1; lower panels show the averaged SMFs during grasping and opening with green and red lines, respectively, and their respective standard errors as shaded areas. Time 0 ms denotes the execution cue time.

Figure 3. Measured cortical activity.

(a,b) Normalized, estimated slow cortical potentials (eSCP) are colour-coded on the normalized brain surface during the attempts of subject 1 to grasp with or open his paralysed right hand (a) or performing the same movement with his intact hand (b). The eSCPs were acquired from 0 to 500 ms relative to the execution cue. R, right; L, left. (c) The one-way ANOVA F-values for the two movements shown in plots (a,b) are colour-coded on the normalized brain surface only for values with significant differences of p < 0.05.

Figure 4. Classification accuracies for different features.

(a,b) Blue and red bars show averaged classification accuracy of movement type (a) or movement intention (b) for affected and intact hands, respectively. Error bars show 95% confidence intervals of classification accuracy. Dotted lines denote chance level. *p < 0.05 and **p < 0.01 significant difference among tested hands (one-way ANOVA with post-hoc Tukey-Kramer).

Figure 5. Offline evaluation of onset detection.

Blue and red lines denote average of first onset detection rate in each time bin for movement of affected and intact hands, respectively. The shaded area shows standard deviation. The N.D. (not detected) bars denote rate of trials in which no onsets were detected. The error bars of the N.D. bars show their standard deviations. For each trial, first onset was searched beginning at −2000 ms. Time 0 ms denotes target time to detect, which is the training time of the class decoder in the training dataset.