The influence of language on perception: listening to sentences about faces affects the perception of faces

Abstract

We examined the effect of linguistic comprehension on early perceptual encoding in a series of electrophysiological and behavioral studies on humans. Using the fact that pictures of faces elicit a robust and reliable evoked response that peaks at ∼170 ms after stimulus onset (N170), we measured the N170 to faces that were preceded by primes that referred to either faces or scenes. When the primes were auditory sentences, the magnitude of the N170 was larger when the face stimuli were preceded by sentences describing faces compared to sentences describing scenes. In contrast, when the primes were visual, the N170 was smaller after visual primes of faces compared to visual primes of scenes. Similar opposing effects of linguistic and visual primes were also observed in a reaction time experiment in which participants judged the gender of faces. These results provide novel evidence of the influence of language on early perceptual processes and suggest a surprising mechanistic description of this interaction: linguistic primes produce content-specific interference on subsequent visual processing. This interference may be a consequence of the natural statistics of language and vision given that linguistic content is generally uncorrelated with the contents of perception.

Figure 1.

a, Timeline of the experimental procedure for sentence priming in experiment 1. Priming sentences, presented through headphones, either referred to faces or scenes. The picture probes were either of a face or a place. Colored fixation crosses (red and blue) prompted each plausibility response. b, Event-related response to face picture probes appearing after face sentence primes (red) and place sentence primes (blue). Shaded areas represent the SE of the difference between the two waveforms. Consequently, the same error term is superimposed on both waveforms.

Figure 2.

a, Timeline of the experimental procedure for picture priming in experiment 2. Primes were either pictures of faces or scenes and were followed by face or place picture probes. b, Event-related response to face picture probes appearing after a face picture primes (red) and place picture primes (blue). Shaded areas represent the SE of the difference between the two waveforms as in Figure 1.

Figure 3.

Peak amplitude of the N170 response to face probes in experiment 1 (left side) and experiment 2 (right side). Within each experiment, the data are plotted for the left (LH) and right (RH) hemispheres, as a function of whether the prime referred to a face or place. Error bars correspond to SE of the difference between the two conditions for each type of prime.

Figure 4.

Reaction time data for experiment 3. Participants made a speeded response, classifying a face probe as male or female. These pictures were preceded by sentence (auditory) or picture (visual) primes that either referred to or depicted a face or place. Error bars correspond to SE of the difference between the two conditions for each type of prime.