Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Abstract

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.

Keywords: EEG; FPVS; Face processing; Face recognition; Famous faces; Frequency tagging.

Figure 1.

a) Experimental design. Participants viewed 60 second blocks consisting of sequentially presented familiar or unfamiliar faces. Faces were presented at either 15 Hz, 6 Hz or 3.75 Hz in upright or inverted orientation. Participants had to count the number of faces that appeared with a green tint (1, 2 or 3 during the block). b) Mean images created from all familiar faces (left) and all unfamiliar faces (right). c) Pixels with significantly higher luminance for unfamiliar > familiar images (left) and familiar > unfamiliar images (right), p < .001, uncorrected.

Figure 2.

Signal to noise ratio (SNR) of periodic signal. a) Mean SNR spectra for upright unfamiliar faces presented at 15 Hz, 6 Hz and 3.75 Hz frequencies. Peaks are evident at the presentation frequencies and their harmonics. b) Head maps showing mean signal for the three flicker frequency conditions (15 Hz, 6 Hz, 3.75 Hz). From left to right, head maps correspond with responses to upright familiar faces, upright unfamiliar faces, inverted familiar faces, and inverted unfamiliar faces at the fundamental frequency. Noise level SNR is 1. Black dots indicate clusters of electrodes with significant signal SNR (above 1). All conditions showed significant signal at the image presentation frequency, ps < .001.

Figure 3.

Differences in SNR between familiar and unfamiliar faces. Black dots indicate clusters of electrodes with significant differences across conditions. At the 15 Hz frequency, SNR was larger for upright familiar faces than for upright unfamiliar faces. At the 6 Hz frequency, SNR was marginally lower for familiar faces. Finally, the 3.75 Hz frequency exhibited no effects of familiarity. There were no differences in SNR for inverted faces at any frequency.

Figure 4.

Multivariate decoding results for 15 Hz frequency. a) Decoding familiar versus unfamiliar faces using searchlight method. b) Whole brain familiarity decoding results as a function of face orientation and participant familiarity ratings. c) Decoding upright versus inverted faces using searchlight method. d) Whole brain orientation decoding results as a function of face familiarity and participant familiarity ratings.

Figure 5.

Multivariate decoding results for 6 Hz frequency. a) Decoding familiar versus unfamiliar faces using searchlight method. Decoding accuracy was significantly above chance for both upright and inverted faces. b) Whole brain familiarity decoding results as a function of face orientation and participant familiarity ratings. c) Decoding upright versus inverted faces using searchlight method. Orientation decoding was better over posterior than frontal clusters. Black dots signify significant clusters of electrodes with above chance decoding. d) Whole brain orientation decoding results as a function of face familiarity and participant familiarity ratings.

Figure 6.

Results for 3.75 Hz frequency. a) Decoding familiar versus unfamiliar faces using searchlight method. Significant decoding was observed for upright, but not inverted, faces. b) Whole brain familiarity decoding results as a function of face orientation and participant familiarity ratings. Participants with high familiarity had significantly better decoding for upright faces. c) Decoding upright versus inverted faces using searchlight method. Black dots signify significant clusters of electrodes with above chance decoding. d) Whole brain orientation decoding results as a function of face familiarity and participant familiarity ratings. Participants with high familiarity had better orientation decoding.