Multimodal Evidence of Atypical Processing of Eye Gaze and Facial Emotion in Children With Autistic Traits

Abstract

According to the shared signal hypothesis (SSH) the impact of facial expressions on emotion processing partially depends on whether the gaze is directed toward or away from the observer. In autism spectrum disorder (ASD) several aspects of face processing have been found to be atypical, including attention to eye gaze and the identification of emotional expressions. However, there is little research on how gaze direction affects emotional expression processing in typically developing (TD) individuals and in those with ASD. This question is investigated here in two multimodal experiments. Experiment 1 required processing eye gaze direction while faces differed in emotional expression. Forty-seven children (aged 9-12 years) participated. Their Autism Diagnostic Observation Schedule (ADOS) scores ranged from 0 to 6 in the experiment. Event-related potentials (ERPs) were sensitive to gaze direction and emotion, but emotion processing did not depend on gaze direction. However, for angry faces the gaze direction effect on the N170 amplitude, as typically observed in TD individuals, diminished with increasing ADOS score. For neutral expressions this correlation was not significant. Experiment 2 required explicit emotion classifications in a facial emotion composite task while eye gaze was manipulated incidentally. A group of 22 children with ASD was compared to a propensity score-matched group of TD children (mean age = 13 years). The same comparison was carried out for a subgroup of nine children with ASD who were less trained in social cognition, according to clinician's report. The ASD group performed overall worse in emotion recognition than the TD group, independently of emotion or gaze direction. However, for disgust expressions, eye tracking data revealed that TD children fixated relatively longer on the eyes of the stimulus face with a direct gaze as compared with averted gaze. In children with ASD we observed no such modulation of fixation behavior as a function of gaze direction. Overall, the present findings from ERPs and eye tracking confirm the hypothesis of an impaired sensitivity to gaze direction in children with ASD or elevated autistic traits, at least for specific emotions. Therefore, we conclude that multimodal investigations of the interaction between emotional processing and stimulus gaze direction are promising to understand the characteristics of individuals differing along the autism trait dimension.

Keywords: ADOS; EPN; N170; autism spectrum disorder; emotion processing; face recognition; gaze direction.

FIGURE 1

Trial scheme of Experiment 1. Presentation of the fixation cross (FIX) for 800 ms, followed by the first picture with one of two emotional expressions and one of three gaze directions (START) for 1 s, and (in most trials) the gaze change (CHANGE) shown for 1 s; a blank screen prompted a button-press decision, whether a change had occurred or not (RESPONSE). The pictures are for illustration only and were neither taken from the CAFE database nor used in the experiment.

FIGURE 2

Grand average ERPs in the START interval. Top: ERPs at electrodes P7 and P8 for neutral and angry expressions. Bottom: Scalp topographies of the emotion effect (angry minus neutral) for the time window of 200–250 and 250–300 ms.

FIGURE 3

Grand average ERPs in the CHANGE interval. Top: ERP waveform of gaze conditions at electrode P7, P8. Bottom: Scalp topographies of the gaze effect (ERPs in the direct to averted condition minus the averted to direct condition) in the time segments 150–190 ms (ns) and 220–270 ms.

FIGURE 4

Correlations between the sum sore of the ADOS scale and the gaze effect to angry (top) and neutral faces (bottom) (data are untransformed; correlation is Spearman). Exclusion of extreme values neither changed the positive slope nor the significance of the correlation (r = 0.32; p < 0.05).

FIGURE 5

Experiment 2. (A) Temporal sequence of a trial in the emotion composite task. Example trial with the target emotion “disgust” at the bottom half of the face. (B) Examples of emotional composite faces. (C) Parameterization of the variables for gaze behavior. The axes show pixels coordinates on the screen. The pictures are for illustration only and were neither taken from the CAFE database nor used in the experiment.

FIGURE 6

Mean performance accuracy of the ASD and TD groups for all emotion conditions and direct and averted gaze of stimulus faces.

FIGURE 7

Emotion composite task. Mean of median vertical gaze positions of the ASD (red) and TD (blue) groups during the first 2 s after stimulus presentation in consecutive time bins of 200 ms.

FIGURE 8

Eye avoidance index (EAI) for the ASD (red) and TD (blue) groups during the first 2 s after stimulus presentation in intervals of 500 ms for all emotions and both gaze directions of the stimulus faces.