Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

Abstract

Recognizing emotion in faces is important in human interaction and survival, yet existing studies do not paint a consistent picture of the neural representation supporting this task. To address this, we collected magnetoencephalography (MEG) data while participants passively viewed happy, angry and neutral faces. Using time-resolved decoding of sensor-level data, we show that responses to angry faces can be discriminated from happy and neutral faces as early as 90 ms after stimulus onset and only 10 ms later than faces can be discriminated from scrambled stimuli, even in the absence of differences in evoked responses. Time-resolved relevance patterns in source space track expression-related information from the visual cortex (100 ms) to higher-level temporal and frontal areas (200-500 ms). Together, our results point to a system optimised for rapid processing of emotional faces and preferentially tuned to threat, consistent with the important evolutionary role that such a system must have played in the development of human social interactions.

Keywords: face perception; magnetoencephalography (MEG); multivariate pattern analysis (MVPA); threat bias.

Figure 1

Experimental paradigm, together with examples of one scrambled image and two face stimuli from the NimStim database, after normalization of Fourier amplitudes [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 2

(a) Sensors exhibiting significant differences to faces compared to scrambled stimuli (marked with asterisks) at the M170 latency (left) and M220 latency (right; p < .01). (b) Timecourses of the evoked responses to neutral faces and scrambled stimuli from right occipital and temporal sensors averaged across subjects (±SEM) [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 3

Searchlight MVPA analysis of differences in face/scrambled stimulus processing. The left‐hand panel summarizes time‐resolved decoding accuracy (averaged across subjects and 50 ms time windows). The right‐hand figure depicts the proportion of participants achieving above‐chance decoding at each sensor regardless of latency (sensors significant in all subjects and selected for further analysis are marked with asterisks) [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 4

(a) Decoding accuracy for the face vs scrambled problem in source space with 95% CI and significant decoding time window (black horizontal line, starting at ∼100 ms). (b) Patterns derived from broadband source‐space decoding of faces and scrambled stimuli for 8 key ROIs for the 0–500 ms time window after stimulus onset. (c) Whole‐brain patterns averaged across the first 250 ms after stimulus onset and plotted on the semi‐inflated MNI template brain. Bilateral ROI labels: CA: calcarine cortex; CU: cuneus; LI: lingual gyrus; OS: occipital superior; OM: occipital medial; OI: occipital inferior; PC: precuneus; FG: fusiform gyrus [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 5

(a) Accuracy traces averaged across participants for each emotion classification problem and each of the four sensor sets (shown in the left‐hand plot). The vertical lines mark the stimulus onset and the shaded areas depict 95% bootstrapped CIs. The horizontal lines represent clusters of at least five significant timepoints (FDR‐corrected p < .05). Significant decoding onset is marked with vertical lines (at ∼100 ms for the angry vs. neutral/happy face decoding using occipital sensors). Accuracy traces were smoothed with a 10‐point moving average for visualization only. (b) As above for the sensor set based on the searchlight feature selection method (shown in the left‐hand plot) [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 6

(a) Accuracy traces averaged across participants for each emotion classification problem in source space using the 84 AAL atlas‐based ROIs (shown in the left‐hand plot). (b) Broadband relevance patterns derived from classifier weights in source space for all three decoding problems, averaged across subjects and 100 ms time windows, baselined and normalized, mapped on the semi‐inflated MNI template brain for time windows between 100 and 500 ms post‐stimulus onset. Patterns visualized here are descriptive and represent each ROI in terms of its relative role in classification across subjects without statistical testing [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 7

Results of permutation testing of relevance patterns shown in Figure 7 for each decoding problem and time window between 100 and 500 ms. Highlighted ROIs were assigned significant weights (p < .05 corrected) [Color figure can be viewed at http://wileyonlinelibrary.com]