Internal representations for face detection: an application of noise-based image classification to BOLD responses

Abstract

What basic visual structures underlie human face detection and how can we extract such structures directly from the amplitude of neural responses elicited by face processing? Here, we address these issues by investigating an extension of noise-based image classification to BOLD responses recorded in high-level visual areas. First, we assess the applicability of this classification method to such data and, second, we explore its results in connection with the neural processing of faces. To this end, we construct luminance templates from white noise fields based on the response of face-selective areas in the human ventral cortex. Using behaviorally and neurally-derived classification images, our results reveal a family of simple but robust image structures subserving face representation and detection. Thus, we confirm the role played by classical face selective regions in face detection and we help clarify the representational basis of this perceptual function. From a theory standpoint, our findings support the idea of simple but highly diagnostic neurally-coded features for face detection. At the same time, from a methodological perspective, our work demonstrates the ability of noise-based image classification in conjunction with fMRI to help uncover the structure of high-level perceptual representations.

Keywords: fMRI; face recognition; reverse correlation.

Figure 1

Base image (left) and examples of the two types of stimuli presented in the experiment: noise‐only (middle) and noise‐plus‐base image (right).

Figure 2

Intermediate results involved in the construction of behaviorally‐derived CIs: four different components corresponding to four types of trials are added to each other in order to estimate the internal template guiding behavioral responses (note that the polarity of μ_FN and μ_NN was flipped for ease of visualization and comparison with the other components). Smoothed components and CIs are displayed under their original (raw) versions. The RMS contrast of each raw image is separately computed for each component and CI. Results are separately shown for subjects EC (top) and EA (bottom).

Figure 3

Procedure for the construction of neurally‐derived CIs: (a) noise fields are grouped and averaged based on ROI response amplitude and stimulus type (F—face base image, N—noise) at a given time point; (b) noise field averages are weighted and combined into a time‐specific CI; (c) a weighted sum is computed across time‐specific CIs using a standard hemodynamic response function (HRF) to generate a single ROI‐specific CI; (d) raw CIs are smoothed with a Gaussian filter to allow analysis and visualization.

Figure 4

Example of ROI mask in subject EA. The map shows the contrast between faces and objects (q < 0.05) superimposed on three axial slices (in EA's native space). The mask is centered on the peak of the right FFA (see Table 1).

Figure 5

Response amplitudes (in percent signal change) across different ROIs as a function of stimulus type and behavioral response (h—hits, fa—false alarms, m—misses, cr—correct rejections). Error bars show ±1 SE across sessions.

Figure 6

Average squared amplitude energy for (a) the base image, stimuli containing the base image and stimuli containing only noise fields; (b) the raw behavioral CIs; (c), (d) the raw neurally‐derived CIs (corresponding to the right FFA and OFA). The abscissa represents spatial frequency in cycles per image and the ordinate displays normalized amplitude values averaged across orientations—values are normalized (scaled) by the maximum value. The average energy of 100 control CIs (constructed by permuting response labels) is shown in gray. Error bars show ±1 SD across stimuli for (a) and across control CIs for (b–d).

Figure 7

Smoothed CIs and their pixel test analysis. Results are shown for (a) behavioral responses, (b) right FFA responses, and (c) the right OFA responses. Blue and yellow mark pixels darker/brighter than chance (P < 0.05).

Figure 8

(a) Symmetrical CIs analyzed with a pixel test (P < 0.05). Results are shown for behavioral CIs (on the left) and for rFFA‐derived CIs (on the right). (b) The two best contrast features for face detection of Viola and Jones [2004] superimposed on a base image.