Hybrid EEG-fNIRS BCI Fusion Using Multi-Resolution Singular Value Decomposition (MSVD)

Abstract

Brain-computer interface (BCI) multi-modal fusion has the potential to generate multiple commands in a highly reliable manner by alleviating the drawbacks associated with single modality. In the present work, a hybrid EEG-fNIRS BCI system-achieved through a fusion of concurrently recorded electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) signals-is used to overcome the limitations of uni-modality and to achieve higher tasks classification. Although the hybrid approach enhances the performance of the system, the improvements are still modest due to the lack of availability of computational approaches to fuse the two modalities. To overcome this, a novel approach is proposed using Multi-resolution singular value decomposition (MSVD) to achieve system- and feature-based fusion. The two approaches based up different features set are compared using the KNN and Tree classifiers. The results obtained through multiple datasets show that the proposed approach can effectively fuse both modalities with improvement in the classification accuracy.

Keywords: EEG; channel selection; classification; fNIRS; hybrid BCI; multi-modal fusion; multi-resolution singular value decomposition.

Figure 1

A four-level filter bank; h[n] is the high pass filter, g[n] is the low pass filter.

Figure 2

A three-level multi-resolution decomposition structure.

Figure 3

MSVD fusion scheme.

Figure 4

Hybrid BCI system using (A) Feature-based fusion. (B) System-based fusion.