Classification for Single-Trial N170 During Responding to Facial Picture With Emotion

Abstract

Whether an event-related potential (ERP), N170, related to facial recognition was modulated by emotion has always been a controversial issue. Some researchers considered the N170 to be independent of emotion, whereas a recent study has shown the opposite view. In the current study, electroencephalogram (EEG) recordings while responding to facial pictures with emotion were utilized to investigate whether the N170 was modulated by emotion. We found that there was a significant difference between ERP trials with positive and negative emotions of around 170 ms at the occipitotemporal electrodes (i.e., N170). Then, we further proposed the application of the single-trial N170 as a feature for the classification of facial emotion, which could avoid the fact that ERPs were obtained by averaging most of the time while ignoring the trial-to-trial variation. In order to find an optimal classifier for emotional classification with single-trial N170 as a feature, three types of classifiers, namely, linear discriminant analysis (LDA), L1-regularized logistic regression (L1LR), and support vector machine with radial basis function (RBF-SVM), were comparatively investigated. The results showed that the single-trial N170 could be used as a classification feature to successfully distinguish positive emotion from negative emotion. L1-regularized logistic regression classifiers showed a good generalization, whereas LDA showed a relatively poor generalization. Moreover, when compared with L1LR, the RBF-SVM required more time to optimize the parameters during the classification, which became an obstacle while applying it to the online operating system of brain-computer interfaces (BCIs). The findings suggested that face-related N170 could be affected by facial expression and that the single-trial N170 could be a biomarker used to monitor the emotional states of subjects for the BCI domain.

Keywords: BCIs; N170; emotional classification; facial recognition; single-trial.

Figure 1

Illustration of facial stimuli. (A) Facial stimuli with three different emotions. (B) An example of the stimulus sequence with emotional pictures.

Figure 2

Behavioral performance analysis. (A) Mean RT with SD. (B) ACC with different emotions. The star denoted that there was a significant difference on RT or ACC between two facial emotions.

Figure 3

The N170 waveforms elicited by facial pictures with positive and negative emotion. (A) Statistical parametric scalp mapping (positive vs. negative). The color bar denoted p-values after performing paired t-test between positive and negative N170. (B) N170 at the left occipitotemporal electrodes. (C) N170 at the right occipitotemporal electrodes. The red lines denoted the N170 waveforms elicited by positive faces, the blue lines denoted the N170 waveforms elicited by negative faces, and the green lines were averaged difference-ERP. The blue star denoted that there was a significant difference between positive and negative N170.