Multi-domain feature joint optimization based on multi-view learning for improving the EEG decoding

Abstract

Background: Brain-computer interface (BCI) systems based on motor imagery (MI) have been widely used in neurorehabilitation. Feature extraction applied by the common spatial pattern (CSP) is very popular in MI classification. The effectiveness of CSP is highly affected by the frequency band and time window of electroencephalogram (EEG) segments and channels selected.

Objective: In this study, the multi-domain feature joint optimization (MDFJO) based on the multi-view learning method is proposed, which aims to select the discriminative features enhancing the classification performance.

Method: The channel patterns are divided using the Fisher discriminant criterion (FDC). Furthermore, the raw EEG is intercepted for multiple sub-bands and time interval signals. The high-dimensional features are constructed by extracting features from CSP on each EEG segment. Specifically, the multi-view learning method is used to select the optimal features, and the proposed feature sparsification strategy on the time level is proposed to further refine the optimal features.

Results: Two public EEG datasets are employed to validate the proposed MDFJO method. The average classification accuracy of the MDFJO in Data 1 and Data 2 is 88.29 and 87.21%, respectively. The classification result of MDFJO was significantly better than MSO (p < 0.05), FBCSP₃₂ (p < 0.01), and other competing methods (p < 0.001).

Conclusion: Compared with the CSP, sparse filter band common spatial pattern (SFBCSP), and filter bank common spatial pattern (FBCSP) methods with channel numbers 16, 32 and all channels as well as MSO, the MDFJO significantly improves the test accuracy. The feature sparsification strategy proposed in this article can effectively enhance classification accuracy. The proposed method could improve the practicability and effectiveness of the BCI system.

Keywords: brain-computer interface; common spatial pattern; electroencephalogram; motor imagery; multi-domain feature joint optimization.

Figure 1

(A) Experimental paradigm and channel arrangement for BCI Competition IV dataset 1. Arrows pointing left, right, or down have been presented as cues for imagining left hand, right hand, or foot movements. After a fixation cross was presented for 2 s, the directional cue was overlaid for 4 s. Then, the screen was blank for 2 s. The number of channels is 59. (B) Experimental paradigm and channel arrangement for BCI Competition III dataset IVa. Within 3.5 s of the visual cues display, the subjects performed the right hand or right foot motor imagery according to the cue. The presentation of target cues is intermitted by periods of random length, 1.75 to 2.25 s, in which the subject could relax. The number of channels is 118. The channel arrangement of Data 1 and Data 2 follows the 10–20 international standard lead system.

Figure 2

FDC value of subject b from Data 1. Channel labels are displayed on each bar.

Figure 3

Multi-view model architecture based on L_2,1. S and T represent the number of channel mode and time intervals, respectively. $V_{s,t}$ represents the CSP feature matrix in the t-th time interval over s-th channel mode. $\mathrm{U}$ is obtained from all views by a solving model. The sparse weight matrix is obtained after the matrix U is regularized by L_2,1-norm regularization.

Figure 4

Feature sparsity strategy demonstration. Q represents the true value of X in a non-zero row coefficient matrix. R means by descending each row in Q.

Figure 5

Framework diagram of the proposed MDFJO for the motor-imagery-related EEG classification. The method mainly includes channel pattern division, sub-band division, and time interval division, feature extracted by CSP and feature selection based on multi-view learning, feature sparsification strategy, and identification of test samples by SVM.

Figure 6

Test results of MDFJO and all comparison algorithms in all subjects (* p < 0.05, ** denotes p < 0.01, *** denotes p < 0.001). The red circle and the blue box represent the test results of all subjects in Data 1 and all subjects in Data 2, respectively.

Figure 7

Three selected channel patterns obtained by using FDC, namely, the 16-32-all channel pattern mode.

Figure 8

Classification accuracy comparison of MDFJO based on different combinations of channel patterns (16-all, 32-all, 16–32, and 16-32-all).

Figure 9

Effect of $\lambda$ change on the selection number of sub-band features and the 5-fold cross-validation accuracy. The left blue ordinate is the 5 × 5 fold accuracy (%), and the right orange coordinate is the number of features. The blue curve and the orange curve, respectively, represent the change in 5 × 5 fold accuracy and the number of features. The red vertical dashed line indicates that when $\lambda$ is 0.5, the accuracy is maximum and the number of features is small.

Figure 10

Influence of N_s on the accuracy of 5-fold cross-validation.

Figure 11

Sparse matrix and average power spectral density of subject f. The left subgraph is a sparse matrix. The darker red color represents a higher weight value. The right subgraph is average power spectral density. Orange and light green lines represent the average power spectral density curves of the two classes. The bottom color bar represents a r² value.

Figure 12

Characteristic difference of each channel mode at each time interval. The red-labeled subgraph indicates that the feature difference is greater than the other time intervals. The red circle and blue cross indicate two types of features.