Resting state EEG assisted imagined vowel phonemes recognition by native and non-native speakers using brain connectivity measures

Abstract

Communication is challenging for disabled individuals, but with advancement of brain-computer interface (BCI) systems, alternative communication systems can be developed. Current BCI spellers, such as P300, SSVEP, and MI, have drawbacks like reliance on external stimuli or conversation irrelevant mental tasks. In contrast to these systems, Imagined speech based BCI systems rely on directly decoding the vowels/words user is thinking, making them more intuitive, user friendly and highly popular among Brain-Computer-Interface (BCI) researchers. However, more research needs to be conducted on how subject-specific characteristics such as mental state, age, handedness, nativeness and resting state activity affects the brain's output during imagined speech. In an overt speech, it is evident that native and non-native speakers' brains function differently. Therefore, this paper explores how nativeness to language affects EEG signals while imagining vowel phonemes, using brain-map analysis and scalogram and also investigates the inclusion of features extracted from resting state EEG with imagined state EEG. The Fourteen-channel EEG for Imagined Speech (FEIS) dataset was used to analyse the EEG signals recorded while imagining vowel phonemes for 16 subjects (nine native English and seven non-native Chinese). For the classification of vowel phonemes, different connectivity measures such as covariance, coherence, and Phase Synchronous Index-PSI were extracted and analysed using statistics based Multivariate Analysis of Variance (MANOVA) approach. Different fusion strategies (difference, concatenation, Common Spatial Pattern-CSP and Canonical Correlation Analysis-CCA) were carried out to incorporate resting state EEG connectivity measures with imagined state connectivity measures for enhancing the accuracy of imagined vowel phoneme recognition. Simulation results revealed that concatenating imagined state and rest state covariance and PSI features provided the maximum accuracy of 92.78% for native speakers and 94.07% for non-native speakers.

Keywords: Connectivity measures; Electroencephalogram; Inter subject/inter trial variability; Nativeness.