The fusiform face area is engaged in holistic, not parts-based, representation of faces

Abstract

Numerous studies with functional magnetic resonance imaging have shown that the fusiform face area (FFA) in the human brain plays a key role in face perception. Recent studies have found that both the featural information of faces (e.g., eyes, nose, and mouth) and the configural information of faces (i.e., spatial relation among features) are encoded in the FFA. However, little is known about whether the featural information is encoded independent of or combined with the configural information in the FFA. Here we used multi-voxel pattern analysis to examine holistic representation of faces in the FFA by correlating spatial patterns of activation with behavioral performance in discriminating face parts with face configurations either present or absent. Behaviorally, the absence of face configurations (versus presence) impaired discrimination of face parts, suggesting a holistic representation in the brain. Neurally, spatial patterns of activation in the FFA were more similar among correct than incorrect trials only when face parts were presented in a veridical face configuration. In contrast, spatial patterns of activation in the occipital face area, as well as the object-selective lateral occipital complex, were more similar among correct than incorrect trials regardless of the presence of veridical face configurations. This finding suggests that in the FFA faces are represented not on the basis of individual parts but in terms of the whole that emerges from the parts.

Figure 1. Stimuli and hypothetical representations.

A) Exemplars of the Veridical face set (top) and the Scrambled face set (bottom). Face stimuli differ either in eyes or mouths within each set. B) Illustration of behavior-neural activation correlational analysis. Correlations of spatial patterns of activation in the FFA were calculated across independent fMRI data sets (i.e., even versus odd runs). We then examined whether the correlation of spatial patterns was higher for correct than incorrect trials. C) “Holistic” representation. If the featural information of faces is combined with the configural information, a higher correlation of spatial patterns in correct trials than incorrect trials shall be observed only when the veridical face configuration is present (i.e., the interaction). D) “Parts-based” representation. If the featural information of faces is encoded independently of the configural information, a higher correlation of spatial patterns in correct trials than incorrect trials shall be observed regardless of whether or not the veridical face configuration is presented (i.e., the main effect). E) “Both” representation. If both types of representations are implemented in the FFA, we shall expect both the interaction and the main effect.

Figure 2. Experimental procedure and behavioral result.

A) Sample trials in the experimental runs. Participants were instructed to discriminate face parts in a sequentially-presented face pair, either both with veridical face configurations (Veridical trials) or both without the configurations (Scrambled trials). The two types of trials were randomly intermixed in the runs. B) Behavioral performance. In the scanner, participants were more accurate (Left) and quicker (Right) in discriminating face parts in the Veridical trials than the Scrambled trials. Error bars indicate standard error of mean (s.e.m.) above and below the mean. *: p<0.001.

Figure 3. The behavioral relevance of the spatial patterns of neural activation in the face-selective regions.

A) Face-selective regions of a typical participant. Face-selective regions, the FFA and OFA, from the fMRI localizer runs are shown on the right hemisphere (RH) of the inflated brain. Sulci are shown in dark gray and gyri in light gray. B) In the FFA, correlations of spatial patterns were higher for correct than incorrect trials only when face parts were presented in a veridical face configuration. * indicates that the two-way interaction of stimulus condition (Veridical versus Scrambled) by response type (Correct versus Incorrect) is significant. C) In the OFA, correlations of spatial patterns were higher for correct than incorrect trials, regardless of configural changes. Error bars indicate s.e.m. above and below the mean. *: p<0.05; n.s.: not significant.

Figure 4. The behavioral relevance of the object-selective regions.

Both the pFs (A) and LO (B) were involved in the parts-based representation of faces, as correlations of spatial patterns of activation in both regions were higher for correct trials than incorrect trials, regardless of configural changes. Error bars indicate s.e.m. above and below the mean.