Plasticity in unimodal and multimodal brain areas reflects multisensory changes in self-face identification

Abstract

Nothing provides as strong a sense of self as seeing one's face. Nevertheless, it remains unknown how the brain processes the sense of self during the multisensory experience of looking at one's face in a mirror. Synchronized visuo-tactile stimulation on one's own and another's face, an experience that is akin to looking in the mirror but seeing another's face, causes the illusory experience of ownership over the other person's face and changes in self-recognition. Here, we investigate the neural correlates of this enfacement illusion using fMRI. We examine activity in the human brain as participants experience tactile stimulation delivered to their face, while observing either temporally synchronous or asynchronous tactile stimulation delivered to another's face on either a specularly congruent or incongruent location. Activity in the multisensory right temporo-parietal junction, intraparietal sulcus, and the unimodal inferior occipital gyrus showed an interaction between the synchronicity and the congruency of the stimulation and varied with the self-reported strength of the illusory experience, which was recorded after each stimulation block. Our results highlight the important interplay between unimodal and multimodal information processing for self-face recognition, and elucidate the neurobiological basis for the plasticity required for identifying with our continuously changing visual appearance.

Keywords: enfacement; fMRI; face recognition; multisensory; self-recognition.

Figure 1.

Experimental design, timeline, and behavioral results. (a) While in the scanner and in 40 s blocks, participants received tactile stimulation to their left cheek from puffs of air. The stimulation was akin to the cotton bud that was seen touching the face of another unfamiliar person in a movie which was played to the participant at the same time. The tactile stimulation on the 2 faces could be either synchronous or asynchronous and on either specularly congruent or incongruent locations. For the incongruent stimulation, participants observed the other person being touched on the chin. After each block of stimulation, participants rated the strength of the illusory experience on a 7-point Likert scale, ranging from “strongly agree” (+3) to “strongly disagree” (−3). (b) The block interval was 50 s, during which there was, a movie (40 s), followed by a blank screen presented for a variable interstimulus interval (0–4 s), followed by the question and Likert scale (maximum 6 s), followed by a blank screen presented for the remaining time to complete 50 s. (c) The mean Likert scale responses across participants for the 4 conditions are shown. As it can be seen, participants showed a stronger illusory experience in the synchronous, congruent condition, as predicted. Error bars depict standard error of the mean.

Figure 2.

fMRI results. Activity in voxels that showed a significant interaction between Synchronicity and Congruency, and also in which activity varied parametrically with the illusory experience. Voxels that showed this response were found in the right TPJ (a), the right IOG (b) and the right IPS (c) and are displayed in the upper panels (P < 0.001 uncorrected is used for display purposes). Plots of the beta coefficients from the peak voxels from the factorial analysis are displayed in the lower panels.