Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Jul 10:9:1128.
doi: 10.3389/fpsyg.2018.01128. eCollection 2018.

A Survey of Automatic Facial Micro-Expression Analysis: Databases, Methods, and Challenges

Yee-Hui Oh  1 , John See  2 , Anh Cat Le Ngo  3 , Raphael C-W Phan  1   4 , Vishnu M Baskaran  5
Affiliations
Review

A Survey of Automatic Facial Micro-Expression Analysis: Databases, Methods, and Challenges

Yee-Hui Oh et al. Front Psychol. .

Abstract

Over the last few years, automatic facial micro-expression analysis has garnered increasing attention from experts across different disciplines because of its potential applications in various fields such as clinical diagnosis, forensic investigation and security systems. Advances in computer algorithms and video acquisition technology have rendered machine analysis of facial micro-expressions possible today, in contrast to decades ago when it was primarily the domain of psychiatrists where analysis was largely manual. Indeed, although the study of facial micro-expressions is a well-established field in psychology, it is still relatively new from the computational perspective with many interesting problems. In this survey, we present a comprehensive review of state-of-the-art databases and methods for micro-expressions spotting and recognition. Individual stages involved in the automation of these tasks are also described and reviewed at length. In addition, we also deliberate on the challenges and future directions in this growing field of automatic facial micro-expression analysis.

Keywords: databases; expressions; facial micro-expressions; recognition; spontaneous; spotting; subtle emotions; survey.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Sample frames from a “Surprise” sequence (Subject 1) in SMIC. Images reproduced from the database with permission from Li et al. (2013).
Figure 2
Figure 2
Sample frames from a “Happiness” sequence (Subject 6) in CASME II. Images reproduced from the database with permission from Yan et al. (2014a).
Figure 3
Figure 3
Sample frames from a “Disgust” sequence (Subject 15) in CAS(ME)2. Images reproduced from the database (©Xiaolan Fu) with permission from Qu et al. (2017).
Figure 4
Figure 4
Sample frames from a sequence (Subject 6) in SAMM that contains micro-movements. Images reproduced from the database with permission from Davison et al. (2016a).
Figure 5
Figure 5
Sample frames from a “Contempt” sequence in MEVIEW that contains micro-movements marked with AU L12. Images reproduced from the database (Husak et al., 2017) under Fair Use.
Figure 6
Figure 6
A video sequence depicting the order in which onset, apex and offset frames occur. Sample frames are from a “Happiness” sequence (Subject 2) in CASME II. Images reproduced from the database with permission from Yan et al. (2014a).
See this image and copyright information in PMC

References

    1. Adegun I. P., Vadapalli H. B. (2016). “Automatic recognition of micro-expressions using local binary patterns on three orthogonal planes and extreme learning machine,” in Pattern Recognition Association of South Africa and Robotics and Mechatronics International Conference (PRASA-RobMech) (Stellenbosch: ), 2016, 1–5.
    1. Allaert B., Bilasco I. M., Djeraba C., Allaert B., Mennesson J., Bilasco I. M., et al. (2017). “Consistent optical flow maps for full and micro facial expression recognition,” in VISAPP, Proc. of the 12th Int. Joint Conf. on Computer Vision, Imaging and Computer Graphics Theory and Applications (Porto: ), 235–242.
    1. Asthana A., Zafeiriou S., Cheng S., Pantic M. (2013). “Robust discriminative response map fitting with constrained local models,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Portland: ), 3444–3451.
    1. Ben X., Jia X., Yan R., Zhang X., Meng W. (2017). Learning effective binary descriptors for micro-expression recognition transferred by macro-information. Pattern Recogn. Lett. 107, 50–58. 10.1016/j.patrec.2017.07.010 - DOI
    1. Bettadapura V. (2012). Face expression recognition and analysis: the state of the art. arXiv preprint arXiv:1203.6722.