Directional Decoding From EEG in a Center-Out Motor Imagery Task With Visual and Vibrotactile Guidance

Abstract

Motor imagery is a popular technique employed as a motor rehabilitation tool, or to control assistive devices to substitute lost motor function. In both said areas of application, artificial somatosensory input helps to mirror the sensorimotor loop by providing kinesthetic feedback or guidance in a more intuitive fashion than via visual input. In this work, we study directional and movement-related information in electroencephalographic signals acquired during a visually guided center-out motor imagery task in two conditions, i.e., with and without additional somatosensory input in the form of vibrotactile guidance. Imagined movements to the right and forward could be discriminated in low-frequency electroencephalographic amplitudes with group level peak accuracies of 70% with vibrotactile guidance, and 67% without vibrotactile guidance. The peak accuracies with and without vibrotactile guidance were not significantly different. Furthermore, the motor imagery could be classified against a resting baseline with group level accuracies between 76 and 83%, using either low-frequency amplitude features or μ and β power spectral features. On average, accuracies were higher with vibrotactile guidance, while this difference was only significant in the latter set of features. Our findings suggest that directional information in low-frequency electroencephalographic amplitudes is retained in the presence of vibrotactile guidance. Moreover, they hint at an enhancing effect on motor-related μ and β spectral features when vibrotactile guidance is provided.

Keywords: brain-computer interface; directional decoding; electroencephalography; kinesthetic guidance; motor imagery; vibrotactile guidance.

Figure 1

Trial structure according to the experimental paradigm, and tactor layout. The time intervals are indicated at the bottom. The middle row illustrates the visual cues, which were identical in both conditions, and the top row (highlighted in green and blue to identify the conditions VtG and noVtG, respectively) the vibrotactile guidance, respectively. The circles represent the tactors, as illustrated in the sketch of the tactor layout on the right. Active tactors are marked in purple, and idle tactors are presented in white. The purple arrows represent movement in both representations (visual and vibrotactile), in the presented example a movement to the right. The participant was visually alerted to the beginning of each trial, 1.5 s before the appearance of the fixation cross. The fixation cross was on screen for 2 s, the latter 1.5 s of which were later used as a baseline period. During this period, participants were instructed to fixate their gaze on the fixation cross, and relax. Afterwards, the visual cue, a right hand with a fixation point, appeared on the monitor. It remained stationary for the pre-MI period of 2 s, and then moved either to the right or up at a constant speed. Participants were instructed to perform the MI in accordance with the movement of the cue. In condition VtG, participants were subsequently asked to judge whether the vibrotactile guidance was congruent to the visual guidance in this trial, and to respond with a key press.

Figure 2

Common preprocessing of EEG signals. The top branch (highlighted in green) shows the main preprocessing applied to the trials, whereas the lower branches show the procedures used to correct for eye artifacts (bottom, orange), and general artifacts (middle, yellow).

Figure 3

Grand-average potentials (0.2–5 Hz), for each condition, and direction. The potentials are represented as 95% confidence intervals of the amplitudes at Cz, CPz, and Pz, complemented by topographic plots at selected time points. The left panel depicts the potentials for condition VtG, whereas the dark green trace corresponds to the direction right, and the light green trace to the direction up, respectively. Similarly, in the left panel, the potentials for direction right are shown in dark blue, and the potentials for direction up in light blue. For time points before the cue movement onset, one set of topographic plots per condition is presented, while for time points after the cue movement onset, one set per condition and direction is presented. These plots are framed with a color-coded frame matching the color of the amplitude traces of the corresponding direction. The sketch in the middle at the bottom highlights the electrode positions whose amplitude traces are shown at the top.

Figure 4

Summary of spectral features. (A) Time-frequency decomposition of the grand-average trial, as time-frequency map, topographic plots, and power spectrum. The time-frequency decomposition and spectrum are computed relative to the period marked by dotted vertical lines in the time-frequency map. The time-frequency map and spectrum are depicted at location C3. The topographic plots were computed from the ranges marked by the black dashed lines in the time-frequency map, i.e., 0.5–2 s in the time dimension, and 8–12 Hz for the μ band, and 15–32 Hz for the β band in the frequency dimension. The bottom panel depicts the grand-average (black solid line) and single-subject spectra (colored dashed lines), respectively, during the MI period (0.5–2 s). The single-subject spectra are grouped into three subgroups, i.e., MI-experienced with stronger than average spectral peaks (yellow), MI-experienced with average or weaker μ peaks (red), and MI-naïve (purple). (B) Power spectra at C3 and topographic plots for the two conditions, and two directions. Green and blue colors identify the conditions. Spectra and topographic plots were computed from the same ranges marked in (A).

Figure 5

Classification results for direction right vs. direction up, based on amplitude features. The grand-average accuracies are depicted as thick solid lines, in green for condition VtG, and in blue for condition noVtG. Single-subject accuracies are represented by dash-dotted dark gray lines, and the individual peak accuracies are marked with squares for MI experienced participants, and diamonds for MI naïve participants, with the same color coding as in Figure 4. Below the accuracy plots, the activation patterns are presented for selected time intervals during the MI period.

Figure 6

Classification results for MI vs. baseline. The distributions of accuracies are presented as box plots, with the individual accuracies identified with the same color coding as in Figures 4, 5. Statistical difference is marked with an asterisk. The activation patterns of the amplitude features, and the power feature CSPs are shown below the box plots.

Figure 7

Classification results for condition VtG vs. condition noVtG, for amplitude features (left), and spectral features (right). The grand-average accuracies are depicted as thick black lines, single-subject accuracies as dash-dotted dark gray lines. The individual peak accuracies are marked with squares for MI experienced participants, and diamonds for MI naïve participants, with the same color coding as in Figures 4–6. Below the accuracy plots, the activation patterns of the amplitude features (bottom left), and the power feature CSPs of the spectral features (bottom right) are presented for selected time intervals within the pre-MI and MI periods.

Figure 8

Overview of subjective ratings on the questionnaire. The (top) row (green colors) refers to condition VtG, the (bottom) row (blue colors) to condition noVtG. Darker colors correspond to a lower rated effort (e.g., “not at all tiring,” or “very easy to concentrate.” The gray portions identify invalid/missing answers.