Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2015 Mar 19;15(3):6719-39.
doi: 10.3390/s150306719.

Multi-layer sparse representation for weighted LBP-patches based facial expression recognition

Qi Jia  1 , Xinkai Gao  2 , He Guo  3 , Zhongxuan Luo  4 , Yi Wang  5
Affiliations

Multi-layer sparse representation for weighted LBP-patches based facial expression recognition

Qi Jia et al. Sensors (Basel). .

Abstract

In this paper, a novel facial expression recognition method based on sparse representation is proposed. Most contemporary facial expression recognition systems suffer from limited ability to handle image nuisances such as low resolution and noise. Especially for low intensity expression, most of the existing training methods have quite low recognition rates. Motivated by sparse representation, the problem can be solved by finding sparse coefficients of the test image by the whole training set. Deriving an effective facial representation from original face images is a vital step for successful facial expression recognition. We evaluate facial representation based on weighted local binary patterns, and Fisher separation criterion is used to calculate the weighs of patches. A multi-layer sparse representation framework is proposed for multi-intensity facial expression recognition, especially for low-intensity expressions and noisy expressions in reality, which is a critical problem but seldom addressed in the existing works. To this end, several experiments based on low-resolution and multi-intensity expressions are carried out. Promising results on publicly available databases demonstrate the potential of the proposed approach.

PubMed Disclaimer

Figures

Figure 1
Figure 1
The basic LBP operator.
Figure 2
Figure 2
LBP feature extracting with weighted patches. (a) Histogram of LBP features; (b) Weighted patches.
Figure 3
Figure 3
Classification result for different testing expressions (a) Angry; (b) Disgust; (c) Fear; (d) Happy; (e) Sad; (f) Surprise.
Figure 3
Figure 3
Classification result for different testing expressions (a) Angry; (b) Disgust; (c) Fear; (d) Happy; (e) Sad; (f) Surprise.
Figure 4
Figure 4
Anger and Disgust in different intensity (a) Anger in high intensity; (b) Disgust in high intensity; (c) Anger in low intensity; (d) Disgust in low intensity.
Figure 5
Figure 5
Multi-layer sparse representation model.
Figure 6
Figure 6
Recognition result of “Fear” with SR and MLSR. (a) “Fear” in low intensity; (b) Recognition result with SR; (c) Recognition result with MLSR.
Figure 7
Figure 7
Comparisons on image resolutions between SR and SVM.
Figure 8
Figure 8
Multi-intensity example of six expressions: Angry, Disgust, Fear, Happy, Sad, Surprise. (a) High intensity; (b) Moderate intensity; (c) Low intensity.
Figure 9
Figure 9
The comparison between MLSR and SR about multi-intensity recognition. (a) High intensity; (b) Moderate intensity; (c) Low intensity.
Figure 10
Figure 10
Facial expression with noise. (a) Anger; (b) Disgust; (c) Sadness.
Figure 11
Figure 11
The comparison between SR and SVM against noise. (a) Angry; (b) Disgust; (c) Sad.
Figure 12
Figure 12
The same expression from different datasets. (a) The CK+ dataset; (b) The JAFFE dataset.
See this image and copyright information in PMC

References

    1. Fang H., Mac Parthalain N., Aubrey A.J., Tam G.K.L., Borgo R., Rosin P.L., Grant P.W., Marshall D., Chen M. Facial expression recognition in dynamic sequences: An integrated approach. Pattern Recogn. 2014;32:740–748.
    1. Zeng Z., Pantic M., Roisman G., Huang T. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 2009;31:39–58. doi: 10.1109/TPAMI.2008.52. - DOI - PubMed
    1. Sorci M., Antonini G., Cruz J., Robin T., Bierlaire M., Thiran J.-P. Modelling human perception of static facial expressions. Image Vis. Comput. 2010;28:790–806. doi: 10.1016/j.imavis.2009.10.003. - DOI
    1. Tian Y. Evaluation of face resolution for expression analysis; Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’04); Washington, DC, USA. 27 May–2 June 2004; pp. 82–88.
    1. Littlewort G., Bartlett M., Fasel I., Susskind J., Movellan J. Dynamics of facial expression extracted automatically from video. Image Vis. Comput. 2006;24:615–625. doi: 10.1016/j.imavis.2005.09.011. - DOI

Publication types