The perception of dynamic and static facial expressions of happiness and disgust investigated by ERPs and fMRI constrained source analysis

Abstract

A recent functional magnetic resonance imaging (fMRI) study by our group demonstrated that dynamic emotional faces are more accurately recognized and evoked more widespread patterns of hemodynamic brain responses than static emotional faces. Based on this experimental design, the present study aimed at investigating the spatio-temporal processing of static and dynamic emotional facial expressions in 19 healthy women by means of multi-channel electroencephalography (EEG), event-related potentials (ERP) and fMRI-constrained regional source analyses. ERP analysis showed an increased amplitude of the LPP (late posterior positivity) over centro-parietal regions for static facial expressions of disgust compared to neutral faces. In addition, the LPP was more widespread and temporally prolonged for dynamic compared to static faces of disgust and happiness. fMRI constrained source analysis on static emotional face stimuli indicated the spatio-temporal modulation of predominantly posterior regional brain activation related to the visual processing stream for both emotional valences when compared to the neutral condition in the fusiform gyrus. The spatio-temporal processing of dynamic stimuli yielded enhanced source activity for emotional compared to neutral conditions in temporal (e.g., fusiform gyrus), and frontal regions (e.g., ventromedial prefrontal cortex, medial and inferior frontal cortex) in early and again in later time windows. The present data support the view that dynamic facial displays trigger more information reflected in complex neural networks, in particular because of their changing features potentially triggering sustained activation related to a continuing evaluation of those faces. A combined fMRI and EEG approach thus provides an advanced insight to the spatio-temporal characteristics of emotional face processing, by also revealing additional neural generators, not identifiable by the only use of an fMRI approach.

Figure 1. Experimental design.

One exemplary trial is depicted including presentation durations in ms of a dynamic and static happy facial expression indicated by a black and grey arrow, respectively. Please note that the subject of the photograph has given written informed consent, as outlined in the PLOS consent form, to publication of her photograph.

Figure 2. Results of ERP and fMRI constrained source analysis of static facial expression.

(A) posthoc comparisons for three selected time windows (dependent t-tests, p<.05). Dashed red boxes represent significant results after FDR-correction. (B) Topographical maps of difference waves for disgust (DIS)>neutral (NEU) and happy (HAP)>NEU and for neutral alone. (C) ERPs for selected posterio-lateral, -midline and anterio-lateral electrode positions. Orange, blue and red boxes denote the analyzed time windows (fN170, EPN, LPP, respectively) including its significant category effects. Orange, blue and red dots in the head model in the upper right corner represent the displayed electrodes of interest and the corresponding time windows. Source model of static stimuli: (D) master grand-average over all electrodes and all conditions (−100–1000 ms), (E) fit of source model indicated by global field power curve (GFP; blue curve) and explained variance of the model (residual variance [RV] and best fit; red curve), (F) Eight RS (each displayed in individual color) projected onto a 4-shell spherical model displayed from 3 different perspectives (saggital [left, right], transversal [above]. Note: RS one to four were seeded based on fMRI activations and RSs five to eight were added by sequential fitting procedure.

Figure 3. Results of ERP and fMRI constrained source analysis of dynamic facial expressions.

(A) posthoc comparisons for six selected time windows (dependent t-tests, p<.05). Dashed red boxes represent significant results after FDR-correction. (B) Topographical maps of difference waves for disgust (DIS)>neutral (NEU) and happiness (HAP)>NEU and neutral alone. (C) ERPs for selected posterior midline and anterio-lateral electrode positions. Red box indicates the analyzed time window of the LPP. Source model of dynamic stimuli: (F) master grand-average over all electrodes and all conditions (−100–1000 ms), (D) Global field power curve (GFP; blue curve) and explained variance of the model (residual variance [RV] and best fit; red curve), (E) Twelve RS (each displayed in individual color) projected onto a 4-shell spherical model displayed from six different perspectives (saggital [left, right], transversal [above, below], coronal [back, front]). Note: RS one to eleven were seeded based on fMRI activations and RS twelve was added by sequential fitting procedure.

Figure 4. Source waveforms of significantly different RS-activations.

Source waveforms (root mean square (RMS) curve of each regional source) are displayed over time (0-1000ms) for each condition (solid line  =  neutrality (NEU), dashed line  =  disgust (DIS), dotted line =  happiness (HAP). Significant RS-activations are based on ANOVAs (light grey: significant at p<.05, dark grey: trend to significance at p<.1) and post hoc comparisons (box: no frame: p<.05, dashed frame: p<.1) for static (A) DIS > NEU, (B) HAP > NEU and dynamic facial expressions (C) DIS > NEU (D) HAP > NEU. Abbreviations: le = left, r = right, CUN  =  cuneus, FUG  =  fusiform gyrus, IFG  =  inferior frontal gyrus, MFG  =  medial frontal gyrus, SFG  =  superior frontal gyrus, TUB  =  tuber, vmPFC  =  ventromedial prefrontal cortex (medial frontal gyrus).