Imagined Speech Classification Using EEG and Deep Learning

Abstract

In this paper, we propose an imagined speech-based brain wave pattern recognition using deep learning. Multiple features were extracted concurrently from eight-channel electroencephalography (EEG) signals. To obtain classifiable EEG data with fewer sensors, we placed the EEG sensors on carefully selected spots on the scalp. To decrease the dimensions and complexity of the EEG dataset and to avoid overfitting during the deep learning algorithm, we utilized the wavelet scattering transformation. A low-cost 8-channel EEG headset was used with MATLAB 2023a to acquire the EEG data. The long-short term memory recurrent neural network (LSTM-RNN) was used to decode the identified EEG signals into four audio commands: up, down, left, and right. Wavelet scattering transformation was applied to extract the most stable features by passing the EEG dataset through a series of filtration processes. Filtration was implemented for each individual command in the EEG datasets. The proposed imagined speech-based brain wave pattern recognition approach achieved a 92.50% overall classification accuracy. This accuracy is promising for designing a trustworthy imagined speech-based brain-computer interface (BCI) future real-time systems. For better evaluation of the classification performance, other metrics were considered, and we obtained 92.74%, 92.50%, and 92.62% for precision, recall, and F1-score, respectively.

Keywords: EEG decoding; LSTM; brain–computer interface (BCI); imagined speech; inner speech; wavelet scattering transformation (WST).

Figure 1

(A) Broca’s and Wernicke’s regions, (B) The electrode positions of the system. Ground and reference are fixed on the back of ears (mastoids) with a disposable sticker, (C) 8-channel EEG headset.

Figure 2

The recording procedure.

Figure 3

Sample of the recorded 8−channel raw EEG dataset at 250 Hz (250 samples per second).

Figure 4

Eight−channel normalized EEG dataset at 250 Hz (250 samples per second).

Figure 5

Eight−extracted features using wavelet scattering transformation at 250 Hz (250 samples per second).

Figure 6

The architecture of the LSTM model.

Figure 7

The data validation and loss curves during the training process of the LSTM model using MATLAB.

Figure 8

The performance of the designed LSTM model. The wrong predicted commands (red bars) were only 6 out of 80 (5 recordings per participant) for all participants, which leads to 92.50% accuracy in the overall prediction of the designed LSTM model.

Figure 9

The confusion matrix for the classification of the four imagined speech commands. The rows represent the predicted class and the columns represent the true class. The diagonal (green) cells correspond to observations that are correctly classified. The off-diagonal (red) cells correspond to incorrectly classified observations. Both the number of observations and the percentage of the total number of observations are shown in each cell. The column on the far right of the plot shows the percentages of all the samples predicted to belong to each class that are correctly and incorrectly classified. The row at the bottom of the plot shows the percentages of all the samples belonging to each class that are correctly and incorrectly classified.