Scanning strategies do not modulate face identification: eye-tracking and near-infrared spectroscopy study

Abstract

Background: During face identification in humans, facial information is sampled (seeing) and handled (processing) in ways that are influenced by the kind of facial image type, such as a self-image or an image of another face. However, the relationship between seeing and information processing is seldom considered. In this study, we aimed to reveal this relationship using simultaneous eye-tracking measurements and near-infrared spectroscopy (NIRS) in face identification tasks.

Methodology/principal findings: 22 healthy adult subjects (8 males and 14 females) were shown facial morphing movies in which an initial facial image gradually changed into another facial image (that is, the subject's own face was changed to a familiar face). The fixation patterns on facial features were recorded, along with changes in oxyhemoglobin (oxyHb) levels in the frontal lobe, while the subjects identified several faces. In the self-face condition (self-face as the initial image), hemodynamic activity around the right inferior frontal gyrus (IFG) was significantly greater than in the familiar-face condition. On the other hand, the scanning strategy was similar in almost all conditions with more fixations on the eyes and nose than on other areas. Fixation time on the eye area did not correlate with changes in oxyHb levels, and none of the scanning strategy indices could estimate the hemodynamic changes.

Conclusions/significance: We conclude that hemodynamic activity, i.e., the means of processing facial information, is not always modulated by the face-scanning strategy, i.e., the way of seeing, and that the right IFG plays important roles in both self-other facial discrimination and self-evaluation.

Figure 1. Conditions of the face identification task.

Directions of each arrow show the course of the morphing movies (starting from the initial image to target image): 1) Self-face condition: from self-face to familiar or unfamiliar face (thick arrows). 2) Familiar-face condition: from familiar face to self or unfamiliar face (dashed arrows). 3) Unfamiliar-face condition: from unfamiliar face to self or familiar face (thin arrows). Both subjects of self- and familiar facial images have given written informed consents (as outlined in the PLoS consent form) to publication of their picture. Unfamiliar facial image was artificially created (nonexistent person).

Figure 2. Time course of one trial.

Subject saw fixation cross for 10 s and the morphing movie started successively. The subject pressed the key when he/she thought that the initial image had changed into the target image and then static noise image appeared for the rest of 20 s. After the movie and noise image, the subject looked at the PC monitor for almost 40 s. We set two baseline data, pre-baseline and post-baseline, to correct the baseline of raw NIRS data with linear fitting.

Figure 3. Location of NIRS probes and channels (a) and Fixation pattern maps and topographies for changes in oxyHb levels in one representative subject (b).

(a) Five emission and four detector probes (gray dots) were arranged in a 3×3 square lattice and the three lowest probes were aligned between Fp1/Fp2 and T3/T4 (blue dots). The mid probe of the three was placed around F7/F8 (international 10/20 system). We obtained cortical responses from a total of 12 channels (red dots) in each hemisphere. Numbers in red dots show channel numbers, i.e. 1 means Ch. 1. (b) Hemodynamic activities in the right hemisphere, especially the areas corresponding with inferior frontal gyrus, were higher in the self-face condition compared with other conditions, but fixation patterns were similar among all conditions. Each line presents a condition: the upper line shows the self-face condition, the middle line shows the familiar face condition, and the lowest line shows the unfamiliar face condition. The left column shows fixation maps and the other two columns show hemodynamic changes, the middle column for the right hemisphere and right column for the left hemisphere. A two-dimensional Gaussian distribution was applied to each of the fixation maps created. The center of this Gaussian distribution is the fixation location and its width is set to 1° of visual angle: the height of the Gaussian is weighted by the proportion of dwell time on each area. Both subjects of self- and familiar facial images have given written informed consents (as outlined in the PLoS consent form) to publication of their picture. Unfamiliar facial image was artificially created (nonexistent person).

Figure 4. Ratios of fixation time in the facial region of interest (fROI) to total fixation time.

Fixation time for the eyes and nose is significantly longer than for the mouth area (p<0.001), though the there is no significant difference between fixation time in each condition. *** p<0.001.

Figure 5. Grand average waveforms in three conditions.

Changes in oxyHb levels in the area corresponding to the right inferior gyrus (right area surrounded by a bold line) were greater in the self-face condition (red wavelike lines) than in the familiar- (green wavelike lines) and unfamiliar- (blue wavelike lines) face conditions. Gray areas indicate the 20-s task period and areas enclosed by bold lines are ROIs for hemodynamic activities (nROI: right for R-IFG; left for L-IFG).

Figure 6. Subtracted changes in oxyHb levels in each nROI.

Changes in oxyHb levels for the unfamiliar-face condition subtracted from the self-face condition (Self – Unfam) show significant increments compared with the unfamiliar-face condition subtracted from the familiar-face condition (Fam – Unfam) in the R-IFG but not in the L-IFG. (R-IFG: right for R; left for L-IFG). *p<0.05.

Figure 7. Relationships between proportions of fixation time in eye area and changes in oxyHb levels in the R-IFG (R-IFG: right for R).

Three scatter graphs show (A) the self-face condition, (B) the familiar-face condition, and (C) the unfamiliar-face condition). None of the correlations reach significance at p<0.05.