A Novel Deep Learning Method Based on an Overlapping Time Window Strategy for Brain-Computer Interface-Based Stroke Rehabilitation

Abstract

Globally, stroke is a leading cause of death and disability. The classification of motor intentions using brain activity is an important task in the rehabilitation of stroke patients using brain-computer interfaces (BCIs). This paper presents a new method for model training in EEG-based BCI rehabilitation by using overlapping time windows. For this aim, three different models, a convolutional neural network (CNN), graph isomorphism network (GIN), and long short-term memory (LSTM), are used for performing the classification task of motor attempt (MA). We conducted several experiments with different time window lengths, and the results showed that the deep learning approach based on overlapping time windows achieved improvements in classification accuracy, with the LSTM combined vote-counting strategy (VS) having achieved the highest average classification accuracy of 90.3% when the window size was 70. The results verified that the overlapping time window strategy is useful for increasing the efficiency of BCI rehabilitation.

Keywords: EEG; brain–computer interface; deep learning method; motor attempt (MA); overlapping time window.

Figure 1

Experimental protocol of rehabilitation training. During the cueing period, a red rectangle is used to alert the user to perform specific tasks. When the cue is a red square, the patient will attempts wrist extension using the stroke-affected hand as hard as possible until the white cross disappears. When the cue is a red rectangle, the patient just needs to stay rested. The patient’s stroke-affected hand was passively extended by the force feedback device when the system accurately identified the patient’s motor intention.

Figure 2

Examples of preprocessed EEG signals from different brain activities.

Figure 3

Losses of the training sets for deep learning models and classification accuracies of test sets for different methods.

Figure 4

The number of parameters and running times for different models.

Figure 5

Accuracies for all methods with different window lengths, each column represents the average of the six method accuracies.

Figure 6

Boxplots represent the ITRs of different sessions for each subject. Each box plot includes 12 sessions of data. The upper and lower lines represent the maximum and minimum ITRs, respectively. The lines in the boxplot represent the median ITR.

Figure 7

Feature visualization of different models for Subject 1. For time window (a), each scatter represents a feature extracted on one trail, and for time window (b), each scatter represents a feature extracted on one time window of one trail.

Figure 8

The confusion matrices for all subjects with the LSTM&VS method.

Figure 9

Visualization of classification results learned by LSTM&VS. In each state, two different sessions (1, 2) of seven subjects are visualized. Each row represents a session and each square represents the correct result for classification in a time window.

Figure 10

The visualization of the averaged topography of the PSD over the alpha band in different time windows. Session 3 with two task types for Subject 3 is visualized.