Role of temporal processing stages by inferior temporal neurons in facial recognition

Abstract

In this review, we focus on the role of temporal stages of encoded facial information in the visual system, which might enable the efficient determination of species, identity, and expression. Facial recognition is an important function of our brain and is known to be processed in the ventral visual pathway, where visual signals are processed through areas V1, V2, V4, and the inferior temporal (IT) cortex. In the IT cortex, neurons show selective responses to complex visual images such as faces, and at each stage along the pathway the stimulus selectivity of the neural responses becomes sharper, particularly in the later portion of the responses. In the IT cortex of the monkey, facial information is represented by different temporal stages of neural responses, as shown in our previous study: the initial transient response of face-responsive neurons represents information about global categories, i.e., human vs. monkey vs. simple shapes, whilst the later portion of these responses represents information about detailed facial categories, i.e., expression and/or identity. This suggests that the temporal stages of the neuronal firing pattern play an important role in the coding of visual stimuli, including faces. This type of coding may be a plausible mechanism underlying the temporal dynamics of recognition, including the process of detection/categorization followed by the identification of objects. Recent single-unit studies in monkeys have also provided evidence consistent with the important role of the temporal stages of encoded facial information. For example, view-invariant facial identity information is represented in the response at a later period within a region of face-selective neurons. Consistent with these findings, temporally modulated neural activity has also been observed in human studies. These results suggest a close correlation between the temporal processing stages of facial information by IT neurons and the temporal dynamics of face recognition.

Keywords: categorization; facial identity; inferior temporal cortex; temporal dynamics.

Figure 1

Time course of two different scales of facial information in the IT cortex. (A) Areas of brain examined in Sugase et al. (1999) (yellow). AP0 represents the position of the external auditory meatus; A14, A19, A22, and A24 represent anterior 14, 19, 22, and 24 mm, respectively. amts, anterior middle temporal sulcus; sts, superior temporal sulcus. (B) Information transmission rate (I_c) of face-responsive neurons. Summation of I_c curves for global (red, human faces vs. monkey faces vs. shapes) and fine categories (black; monkey facial expression, monkey identity, human facial expression, or human identity) from 32 neurons aligned at the stimulus onset. I_c was calculated over a 50-ms sliding analysis window, and plotted at the middle of the time window. Non-significant information was excluded from the analysis. The 32 neurons represent information about both global and fine categories. In the time axis (abscissa), the time of stimulus onset is 0 ms and the duration of stimulus presentation is indicated as a dashed line. (C) Population activity vectors in two-dimensional space rearranged using PCA in the [90, 140] and [140, 190]-ms windows. The horizontal axis represents the first principal component, while the vertical axis represents the second principal component. The points indicate the population activity vectors of 45 face-responsive neurons for the individual stimuli. The colors of the points represent the global category: the vectors for human faces, monkey faces, and shapes are shown in red, blue, and green, respectively. The red, blue, and green ellipses indicate the distributions of the population activity vectors for the fine categories human identity, monkey expression, and shape-form, respectively. Figures adapted from Sugase et al. (1999) (A,B) and Matsumoto et al. (2005b) (C) with permission.

Figure 2

Time course of information about facial identity across different facial views in face patches. (A) Face selectivity in different parts of the macaque temporal lobe. Inflated macaque left hemisphere (dark gray areas mark sulci, light gray–dark gray boundaries mark the middle of the bank within a sulcus) showing six regions, i.e., face patches, in the temporal lobe that responded significantly more to faces than to objects in fMRI experiments. Color scale indicates negative common logarithm of the P value. The six patches are indicated as PL, posterior lateral; ML, middle lateral; MF, middle fundus; AF, anterior fundus; AL, anterior lateral; and AM, anterior medial patch. (B) Emergence of view-invariant identity-selectivity over time. View-invariant identity-selectivity index, computed over a 200-ms sliding response window beginning at the indicated time point, plotted for AM, AL, and ML/MF (solid curves in magenta, blue, and green, respectively). The dotted curves show the mean view-invariant identity-selectivity index over time computed after the relationships between the face view and the face identity are shuffled. The grayscale traces show the time course of the mean response across the population in each face patch. There is the substantial delay between the peak of the mean response to the stimuli and the peak of the view-invariant identity-selectivity index. Figures adapted from Freiwald and Tsao (2010) with permission.