Electrophysiological Indices of Audiovisual Speech Perception in the Broader Autism Phenotype

Abstract

When a speaker talks, the consequences of this can both be heard (audio) and seen (visual). A novel visual phonemic restoration task was used to assess behavioral discrimination and neural signatures (event-related potentials, or ERP) of audiovisual processing in typically developing children with a range of social and communicative skills assessed using the social responsiveness scale, a measure of traits associated with autism. An auditory oddball design presented two types of stimuli to the listener, a clear exemplar of an auditory consonant-vowel syllable /ba/ (the more frequently occurring standard stimulus), and a syllable in which the auditory cues for the consonant were substantially weakened, creating a stimulus which is more like /a/ (the infrequently presented deviant stimulus). All speech tokens were paired with a face producing /ba/ or a face with a pixelated mouth containing motion but no visual speech. In this paradigm, the visual /ba/ should cause the auditory /a/ to be perceived as /ba/, creating an attenuated oddball response; in contrast, a pixelated video (without articulatory information) should not have this effect. Behaviorally, participants showed visual phonemic restoration (reduced accuracy in detecting deviant /a/) in the presence of a speaking face. In addition, ERPs were observed in both an early time window (N100) and a later time window (P300) that were sensitive to speech context (/ba/ or /a/) and modulated by face context (speaking face with visible articulation or with pixelated mouth). Specifically, the oddball responses for the N100 and P300 were attenuated in the presence of a face producing /ba/ relative to a pixelated face, representing a possible neural correlate of the phonemic restoration effect. Notably, those individuals with more traits associated with autism (yet still in the non-clinical range) had smaller P300 responses overall, regardless of face context, suggesting generally reduced phonemic discrimination.

Keywords: ERP; audiovisual speech perception; broader autism phenotype; development.

Figure A1

Frequency distribution of useable trials as a function of face context and speech stimulus.

Figure 1

Spectrogram of /ba/ and /a/ synthesized auditory speech stimuli. First panel, top left, spectrogram of synthesized /ba/; Second panel, top right, edited synthesized /ba/ with reduced initial formants for the consonant, referred to as /a/.

Figure 2

Image of audiovisual (AV) and pixelated (PX) face condition stimuli. Left panel audiovisual face condition, showing the visible articulation of the speaker; Right panel pixelated face condition, showing the speaker’s face, but obscuring the mouth.

Figure 3

Waveform plots showing the N100 and P300. (A): electrode montage; (B) and (C): N100 and P300 response to standard /ba/ and deviant /a/ in the audiovisual (AV) condition; Right panel: N100 and P300 response to standard /ba/ and deviant /a/ in the pixelated video (PX) condition. Shading around waveforms represents the standard error from the mean.

Figure 4

Topomaps for the N100 and P300 effects by condition.

Figure 5

Scatter plots showing the correlation between the late P300 effect and the social responsiveness scale (SRS) total score in the AV and PX conditions.