Imitation and recognition of facial emotions in autism: a computer vision approach

Abstract

Background: Imitation of facial expressions plays an important role in social functioning. However, little is known about the quality of facial imitation in individuals with autism and its relationship with defining difficulties in emotion recognition.

Methods: We investigated imitation and recognition of facial expressions in 37 individuals with autism spectrum conditions and 43 neurotypical controls. Using a novel computer-based face analysis, we measured instructed imitation of facial emotional expressions and related it to emotion recognition abilities.

Results: Individuals with autism imitated facial expressions if instructed to do so, but their imitation was both slower and less precise than that of neurotypical individuals. In both groups, a more precise imitation scaled positively with participants' accuracy of emotion recognition.

Limitations: Given the study's focus on adults with autism without intellectual impairment, it is unclear whether the results generalize to children with autism or individuals with intellectual disability. Further, the new automated facial analysis, despite being less intrusive than electromyography, might be less sensitive.

Conclusions: Group differences in emotion recognition, imitation and their interrelationships highlight potential for treatment of social interaction problems in individuals with autism.

Keywords: Autism; Automated analysis; Emotion recognition; Facial expression; Imitation.

Fig. 1

Example pictures of BERT for each emotion

Fig. 2

Time course of a trial of BERT for both conditions

Fig. 3

Automated facial analysis of imitation. At the left, the automated facial analysis of the stimulus material is displayed. At the right, the automated facial analysis of a neurotypical subject (*represented by the experimenter) in the imitation condition is displayed. Both measures are combined to analyze the imitation as described in the method section

Fig. 4

Calculation of cosine similarity between participant’s and actor’s vector

Fig. 5

a Comparison of intensity of most similar expression in the imitation condition separated by emotions. b Comparison of intensity of most similar expression in the imitation condition for neurotypical individuals (left) and individuals with autism (right) separated by emotions

Fig. 6

Comparison of intensity of most similar expression in the imitation condition for neurotypical individuals (left) and individuals with autism (right)

Fig. 7

Comparison of timepoint (i.e. number of frames) of maxima of relevant action units during imitation for neurotypical individuals (left) and individuals with autism (right)