Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces - Current Limitations and Future Directions

Abstract

Multi-modal integration, which combines multiple neurophysiological signals, is gaining more attention for its potential to supplement single modality's drawbacks and yield reliable results by extracting complementary features. In particular, integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) is cost-effective and portable, and therefore is a fascinating approach to brain-computer interface (BCI). However, outcomes from the integration of these two modalities have yielded only modest improvement in BCI performance because of the lack of approaches to integrate the two different features. In addition, mismatch of recording locations may hinder further improvement. In this literature review, we surveyed studies of the integration of EEG/fNIRS in BCI thoroughly and discussed its current limitations. We also suggested future directions for efficient and successful multi-modal integration of EEG/fNIRS in BCI systems.

Keywords: brain-computer interface (BCI); electroencephalography (EEG); functional near-infrared spectroscopy (fNIRS); multi-modal integration.

FIGURE 1

Commercial EEG-fNIRS devices: (A) fNIRS-EEG package (Courtesy of Artinis Medical Systems, Netherlands, http://www.artinis.com, up to 112-ch fNIRS and 128-ch EEG); (B) LABNIRS (Courtesy of Shimadzu Corporation, Japan, http://www.shimadzu.com, up to 142-ch fNIRS and 64-ch EEG); (C) NIRScout (Courtesy of NIRx Medical Technologies, http://nirx.net, Up to 182-ch fNIRS and 32-ch EEG) created by Buccino et al. (2016).