Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 May 11;11(1):31.
doi: 10.1186/s13229-020-00327-4.

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

Abigail Bangerter  1 , Meenakshi Chatterjee  2 , Joseph Manfredonia  3 , Nikolay V Manyakov  4 , Seth Ness  3 , Matthew A Boice  3 , Andrew Skalkin  3 , Matthew S Goodwin  5 , Geraldine Dawson  6 , Robert Hendren  7 , Bennett Leventhal  7 , Frederick Shic  8   9 , Gahan Pandina  3
Affiliations

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

Abigail Bangerter et al. Mol Autism. .

Abstract

Background: Reduction or differences in facial expression are a core diagnostic feature of autism spectrum disorder (ASD), yet evidence regarding the extent of this discrepancy is limited and inconsistent. Use of automated facial expression detection technology enables accurate and efficient tracking of facial expressions that has potential to identify individual response differences.

Methods: Children and adults with ASD (N = 124) and typically developing (TD, N = 41) were shown short clips of "funny videos." Using automated facial analysis software, we investigated differences between ASD and TD groups and within the ASD group in evidence of facial action unit (AU) activation related to the expression of positive facial expression, in particular, a smile.

Results: Individuals with ASD on average showed less evidence of facial AUs (AU12, AU6) relating to positive facial expression, compared to the TD group (p < .05, r = - 0.17). Using Gaussian mixture model for clustering, we identified two distinct distributions within the ASD group, which were then compared to the TD group. One subgroup (n = 35), termed "over-responsive," expressed more intense positive facial expressions in response to the videos than the TD group (p < .001, r = 0.31). The second subgroup (n = 89), ("under-responsive"), displayed fewer, less intense positive facial expressions in response to videos than the TD group (p < .001; r = - 0.36). The over-responsive subgroup differed from the under-responsive subgroup in age and caregiver-reported impulsivity (p < .05, r = 0.21). Reduced expression in the under-responsive, but not the over-responsive group, was related to caregiver-reported social withdrawal (p < .01, r = - 0.3).

Limitations: This exploratory study does not account for multiple comparisons, and future work will have to ascertain the strength and reproducibility of all results. Reduced displays of positive facial expressions do not mean individuals with ASD do not experience positive emotions.

Conclusions: Individuals with ASD differed from the TD group in their facial expressions of positive emotion in response to "funny videos." Identification of subgroups based on response may help in parsing heterogeneity in ASD and enable targeting of treatment based on subtypes.

Trial registration: ClinicalTrials.gov, NCT02299700. Registration date: November 24, 2014.

Keywords: Autism spectrum disorder; Emotional regulation; Emotions; Facial expression; Impulsive behavior.

PubMed Disclaimer

Conflict of interest statement

AB, JM, MC, SN, MB, NVM, and GP are employees of Janssen Research & Development, LLC, and may hold company equity. AS was an employee of Janssen Research & Development at the time of the study. MSG has received research and consulting funding from Janssen Research & Development. GD is on the Scientific Advisory Boards of Janssen Research & Development; Akili, Inc.; LabCorp, Inc.; and Roche Pharmaceutical Company; is a consultant for Apple, Inc; Gerson Lehrman Group; Guidepoint, Inc.; and Axial Ventures; has received grant funding from Janssen Research & Development; and is the CEO of DASIO, LLC. GD receives royalties from Guilford Press, Springer, and Oxford University Press. RH received reimbursement for consultation from Janssen Research & Development. BL has received research grant funding from the NIH; is a consultant to Janssen Research & Development, the Illinois Children’s Healthcare Foundation; and is a board member of the Brain Research Foundation. FS is on the Scientific Advisory Board, is a consultant to and received grant funding from Janssen Research & Development, and has also received grant funding from Roche.

Figures

Fig. 1
Fig. 1
Plot of average AU12 between TD and ASD groups. ASD, autism spectrum disorder; AU, action unit; TD typically developing
Fig. 2
Fig. 2
Scatterplot of average AU6 and average AU12. Red and blue dots represent average AU6 and average AU12 for over-responsive and under-responsive subgroups, respectively. The average AU6 and average AU12 values of participants from TD group are overlaid as green dots. ASD, autism spectrum disorder; AU, action unit; TD, typically developing
Fig. 3
Fig. 3
Plot of AU12 AUC (a), AU6 AUC (b), average AU12 (c), and average AU6 (d) between ASD subgroups and TD. ASD, autism spectrum disorder; AU, action unit; AUC, area under the curve; TD, typically developing
Fig. 4
Fig. 4
Plot of ABI SR Impulsivity (top) and age (bottom) between ASD subgroups. ASD, autism spectrum disorder; AU, action unit; AUC, area under the curve
See this image and copyright information in PMC

References

    1. American Psychiatric Association . Diagnostic and statistical manual of mental disorders, 5th edition: DSM-5. Arlinglton, TX: American Psychiatric Publishing; 2013.
    1. Begeer S, Koot HM, Rieffe C, Meerum Terwogt M, Stegge H. Emotional competence in children with autism: diagnostic criteria and empirical evidence. Dev Rev. 2008;28(3):342–369. doi: 10.1016/j.dr.2007.09.001. - DOI
    1. Davies H, Wolz I, Leppanen J, Fernandez-Aranda F, Schmidt U, Tchanturia K. Facial expression to emotional stimuli in non-psychotic disorders: a systematic review and meta-analysis. Neurosci Biobehav Rev. 2016;64:252–271. doi: 10.1016/j.neubiorev.2016.02.015. - DOI - PubMed
    1. McIntosh DN, Reichmann-Decker A, Winkielman P, Wilbarger JL. When the social mirror breaks: deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Dev Sci. 2006;9(3):295–302. doi: 10.1111/j.1467-7687.2006.00492.x. - DOI - PubMed
    1. Stagg SD, Slavny R, Hand C, Cardoso A, Smith P. Does facial expressivity count? How typically developing children respond initially to children with autism. Autism. 2014;18(6):704–711. doi: 10.1177/1362361313492392. - DOI - PubMed

Publication types

Associated data