Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Apr 21;19(8):1897.
doi: 10.3390/s19081897.

Recognition of Emotion Intensities Using Machine Learning Algorithms: A Comparative Study

Dhwani Mehta  1 , Mohammad Faridul Haque Siddiqui  2 , Ahmad Y Javaid  3
Affiliations

Recognition of Emotion Intensities Using Machine Learning Algorithms: A Comparative Study

Dhwani Mehta et al. Sensors (Basel). .

Abstract

Over the past two decades, automatic facial emotion recognition has received enormous attention. This is due to the increase in the need for behavioral biometric systems and human-machine interaction where the facial emotion recognition and the intensity of emotion play vital roles. The existing works usually do not encode the intensity of the observed facial emotion and even less involve modeling the multi-class facial behavior data jointly. Our work involves recognizing the emotion along with the respective intensities of those emotions. The algorithms used in this comparative study are Gabor filters, a Histogram of Oriented Gradients (HOG), and Local Binary Pattern (LBP) for feature extraction. For classification, we have used Support Vector Machine (SVM), Random Forest (RF), and Nearest Neighbor Algorithm (kNN). This attains emotion recognition and intensity estimation of each recognized emotion. This is a comparative study of classifiers used for facial emotion recognition along with the intensity estimation of those emotions for databases. The results verified that the comparative study could be further used in real-time behavioral facial emotion and intensity of emotion recognition.

Keywords: automatic facial emotion recognition; behavioral biometrical systems; intensity of emotion recognition; machine learning.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Illustration of 5 Basic Emotions: Happy, Surprise, Neutral, Sad and Angry.
Figure 2
Figure 2
Generalized Architecture for Intensity of Emotion Recognition.
Figure 3
Figure 3
Relation between scale of evidence and intensities of facial action units.
Figure 4
Figure 4
AU Combination: (A) AU12 occurs alone; (B) AU15 occurs alone; (C) AU12 and AU15 occur together—non-additive.
Figure 5
Figure 5
AU Combination: (Case A) AU6 + AU12 + AU25, (Case B) AU4 + AU15 + AU17.
Figure 6
Figure 6
Comparison of AU intensity Labels on Database.
Figure 7
Figure 7
Comparison of AU intensity Labels on Database.
Figure 8
Figure 8
Comparison of AU intensity Labels on Database.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Wang L. Behavioral Biometrics for Human Identification: Intelligent Applications: Intelligent Applications. IGI Global; Hershey, PA, USA: 2009.
    1. Gamboa H., Fred A. A behavioral biometric system based on human-computer interaction. Proc. SPIE. 2004;5404:381–393.
    1. Hess U., Banse R., Kappas A. The intensity of facial expression is determined by underlying affective state and social situation. J. Personal. Soc. Psychol. 1995;69:280. doi: 10.1037/0022-3514.69.2.280. - DOI
    1. Bronstein A.M., Bronstein M.M., Kimmel R. International Conference on Audio-and Video-Based Biometric Person Authentication. Springer; Berlin/Heidelberg, Germany: 2003. Expression-invariant 3D face recognition; pp. 62–70.
    1. Hesher C., Srivastava A., Erlebacher G. A novel technique for face recognition using range imaging; Proceedings of the Seventh International Symposium on Signal Processing and Its Applications; Paris, France. 4 July 2003; pp. 201–204.

LinkOut - more resources