Affective state estimation based on Russell's model and physiological measurements

Abstract

Affective states are psycho-physiological constructs connecting mental and physiological processes. They can be represented in terms of arousal and valence according to the Russel's model and can be extracted from physiological changes in human body. However, a well-established optimal feature set and a classification method effective in terms of accuracy and estimation time are not present in the literature. This paper aims at defining a reliable and efficient approach for real-time affective state estimation. To obtain this, the optimal physiological feature set and the most effective machine learning algorithm, to cope with binary as well as multi-class classification problems, were identified. ReliefF feature selection algorithm was implemented to define a reduced optimal feature set. Supervised learning algorithms, such as K-Nearest Neighbors (KNN), cubic and gaussian Support Vector Machine, and Linear Discriminant Analysis, were implemented to compare their effectiveness in affective state estimation. The developed approach was tested on physiological signals acquired on 20 healthy volunteers during the administration of images, belonging to the International Affective Picture System, conceived for inducing different affective states. ReliefF algorithm reduced the number of physiological features from 23 to 13. The performances of machine learning algorithms were compared and the experimental results showed that both accuracy and estimation time benefited from the optimal feature set use. Furthermore, the KNN algorithm resulted to be the most suitable for affective state estimation. The results of the assessment of arousal and valence states on 20 participants indicate that KNN classifier, adopted with the 13 identified optimal features, is the most effective approach for real-time affective state estimation.

Figure 1

Main modules of the experimental workflow: Graphical User Interface (GUI) and Physiological Monitoring Module.

Figure 2

Valence-arousal plane divided into 5 quadrants: HAHV, HALV, LALV, LAHV, and N. The neutral region N is located in the center of the valence-arousal plane. The colored dots represent the location of the 50 IAPS images considered in this work.

Figure 3

Experimental protocol: order of administration of visual stimuli by the GUI, organized in 10 different clusters of 5 IAPS images each.

Figure 4

Boxplots of the weights for each feature provided by the use of the ReliefF algorithm for k = 5. The dashed line establishes the threshold for identifying the relevant features.

Figure 5

Accuracy of the machine learning algorithms estimation before (blue bars) and after (orange bars) feature selection using ReliefF algorithm. Stars highlight the models that significantly improve their performance after ReliefF.

Figure 6

Machine learning algorithms accuracies of the four classification problems (i.e. H/L valence, H/L arousal, 4 classes and 5 classes) using the optimal feature set. Significant differences between machine learning algorithms are also highlighted with black stars.

Figure 7

Mean estimation times using all 23 extracted features and the optimal feature set achieved with the ReliefF algorithm.