Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

Abstract

The use of imagined speech with electroencephalographic (EEG) signals is a promising field of brain-computer interfaces (BCI) that seeks communication between areas of the cerebral cortex related to language and devices or machines. However, the complexity of this brain process makes the analysis and classification of this type of signals a relevant topic of research. The goals of this study were: to develop a new algorithm based on Deep Learning (DL), referred to as CNNeeg1-1, to recognize EEG signals in imagined vowel tasks; to create an imagined speech database with 50 subjects specialized in imagined vowels from the Spanish language (/a/,/e/,/i/,/o/,/u/); and to contrast the performance of the CNNeeg1-1 algorithm with the DL Shallow CNN and EEGNet benchmark algorithms using an open access database (BD1) and the newly developed database (BD2). In this study, a mixed variance analysis of variance was conducted to assess the intra-subject and inter-subject training of the proposed algorithms. The results show that for intra-subject training analysis, the best performance among the Shallow CNN, EEGNet, and CNNeeg1-1 methods in classifying imagined vowels (/a/,/e/,/i/,/o/,/u/) was exhibited by CNNeeg1-1, with an accuracy of 65.62% for BD1 database and 85.66% for BD2 database.

Keywords: brain-computer interface (BCI); convolutional neural networks (CNN); deep learning; electroencephalography; imagined speech; vowels.

Figure A1

Reference architectures with CNN Shallow CNN.

Figure A2

Reference architectures with EEGNet.

Figure 1

Location of the neuroheadset, which contains 14 electrodes (E1,…,E14) covering a section of the left hemisphere (language area). This is, the sensorimotor interface area and articulatory network of Hickok and Poeppel (Broca’s area and motor cortex) related to Brodmann areas: 4, 6, 43, 44 and 45. C3 and T3 are reference points from the 10–20 positioning system.

Figure 2

Time intervals for the imagined vowels experiment. When the light is on, the subject imagines the vowel and when the light is off, the subject relaxes.

Figure 3

Application of the APIT-MEMD algorithm to a trial of imagined vowel signals (/a/,/e/,/i/,/o/,/u/), where the first two IMFs (IMF1 and IMF2) are shown in the time-domain (blue) and the frequency-domain (red) for a subject in BD2.

Figure 4

CNNeeg1-1 architecture made up of 10 specialized CNNs and a one-against-one function.

Figure 5

Intra-subject training classification accuracy for the Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD1 database.

Figure 6

Intra-subject training classification accuracy for the Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD2 database.

Figure 7

Marginal measures in intra-subject training classification for Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD1 and BD2 databases.

Figure 8

Subject’s Internal representation (CAM) in imagined tasks of the vowel /a/. The horizontal axis represents the difference between electrode and the vertical axis, the corresponding frequencies.

Figure 9

Subject’s internal representation (CAM) in imagined tasks of the vowel /e/. The horizontal axis represents, the difference between electrodes and the vertical axis, the corresponding frequencies.

Figure 10

Subject’s internal representation (CAM) in imagined tasks of the vowel /i/. The horizontal axis represents the difference between electrodes and the vertical axis, the corresponding frequencies.

Figure 11

Subject’s internal representation (CAM) in imagined tasks of the vowel /o/. The horizontal axis represents the difference between electrodes and the vertical axis, the corresponding frequencies.

Figure 12

Subjects’ internal representation (CAM) in imagined tasks of the vowel /u/. The horizontal axis represents the difference between electrodes and the vertical axis, the corresponding frequencies.

Figure 13

Inter-subject training classification accuracy for the Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD1 database.

Figure 14

Inter-subject training classification accuracy for the Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD2 database.

Figure 15

Marginal measures in the inter-subject training classification for the Shallow CNN, EEGNet, and CNNeeg1-1 algorithms using BD1 and BD2 databases.