Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Aug 10;23(16):7092.
doi: 10.3390/s23167092.

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review

Andrada-Livia Cîrneanu  1 , Dan Popescu  1 , Dragoș Iordache  2
Affiliations

New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review

Andrada-Livia Cîrneanu et al. Sensors (Basel). .

Abstract

Facial emotion recognition (FER) is a computer vision process aimed at detecting and classifying human emotional expressions. FER systems are currently used in a vast range of applications from areas such as education, healthcare, or public safety; therefore, detection and recognition accuracies are very important. Similar to any computer vision task based on image analyses, FER solutions are also suitable for integration with artificial intelligence solutions represented by different neural network varieties, especially deep neural networks that have shown great potential in the last years due to their feature extraction capabilities and computational efficiency over large datasets. In this context, this paper reviews the latest developments in the FER area, with a focus on recent neural network models that implement specific facial image analysis algorithms to detect and recognize facial emotions. This paper's scope is to present from historical and conceptual perspectives the evolution of the neural network architectures that proved significant results in the FER area. This paper endorses convolutional neural network (CNN)-based architectures against other neural network architectures, such as recurrent neural networks or generative adversarial networks, highlighting the key elements and performance of each architecture, and the advantages and limitations of the proposed models in the analyzed papers. Additionally, this paper presents the available datasets that are currently used for emotion recognition from facial expressions and micro-expressions. The usage of FER systems is also highlighted in various domains such as healthcare, education, security, or social IoT. Finally, open issues and future possible developments in the FER area are identified.

Keywords: artificial intelligence; deep learning; facial emotion recognition; neural network.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Main components of a facial emotions recognition system based on ANN.
Figure 2
Figure 2
SCOPUS and Web of Science search results on keywords between 2018 and 2022: neural networks, deep learning, emotion recognition, images classification, separately.
Figure 3
Figure 3
SCOPUS and Web of Science search results on combined keywords between 2018 and 2022.
Figure 4
Figure 4
PRISMA flow diagram of the research.
Figure 5
Figure 5
CNN architecture.
Figure 6
Figure 6
GAN architecture.
Figure 7
Figure 7
RNN architecture.
See this image and copyright information in PMC

References

    1. Sapiński T., Kamińska D., Pelikant A., Anbarjafari G. Emotion Recognition from Skeletal Movements. Entropy. 2019;21:646. doi: 10.3390/e21070646. - DOI - PMC - PubMed
    1. Ekman P., Freisen W., Ancoli S. Facial signs of emotional experience. J. Personal. Soc. Psychol. 1980;39:1125. doi: 10.1037/h0077722. - DOI
    1. Ekman P., Friesen W.V. Environmental Psychology & Nonverbal Behavior. American Psychological Association; Washington, DC, USA: 1978. Facial action coding system.
    1. Barrett L.F. The theory of constructed emotion: An active inference account of interoception and categorization. Soc. Cogn. Affect. Neurosci. 2017;12:1–23. doi: 10.1093/scan/nsx060. - DOI - PMC - PubMed
    1. Küntzler T., Höfling T.T.A., Alpers G.W. Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions. Front. Psychol. 2021;12:627561. doi: 10.3389/fpsyg.2021.627561. - DOI - PMC - PubMed

Publication types

LinkOut - more resources