Neural mechanisms for illusory filling-in of degraded speech

Abstract

The brain uses context and prior knowledge to repair degraded sensory inputs and improve perception. For example, listeners hear speech continuing uninterrupted through brief noises, even if the speech signal is artificially removed from the noisy epochs. In a functional MRI study, we show that this temporal filling-in process is based on two dissociable neural mechanisms: the subjective experience of illusory continuity, and the sensory repair mechanisms that support it. Areas mediating illusory continuity include the left posterior angular gyrus (AG) and superior temporal sulcus (STS) and the right STS. Unconscious sensory repair occurs in Broca's area, bilateral anterior insula, and pre-supplementary motor area. The left AG/STS and all the repair regions show evidence for word-level template matching and communicate more when fewer acoustic cues are available. These results support a two-path process where the brain creates coherent perceptual objects by applying prior knowledge and filling-in corrupted sensory information.

Fig.1. Stimuli

The time waveforms (top) and corresponding spectrograms (bottom) for an example word (”aggressor”). The middle panel depicts a physically interrupted word, where white noise replaced part of the fricative/ss/. The right panel depicts a physically continuous word, where white noise was superimposed on the fricative. The left panel depicts the original word with no noise, which was not among the experimental stimuli.

Fig. 2. Stimulus properties equalized across conditions

Number of natural, illusion, and illusion-failure responses according to white noise burst duration. Duration is relative to the distorted speech sound, quantified as the proportion of the entire phoneme interrupted by (illusion and illusion-failure) or added to (natural) white noise. On the x-axis, the white noise proportion for each of the conditions is normalized to the average white noise proportion of the three conditions (142% of the fricative).

Fig. 3. Hypothesis

A schematic depicting the predicted processing paths mediating illusory filling-in of speech. Our design explicitly tests whether the continuity and repair pathways are neurally dissociable. Contrasting BOLD activity for natural > illusion identifies the repair network, and contrasting illusion > illusion-failure identifies the continuity network. See text for full description.

Fig. 4. Dissociable networks for repair and continuity

A. Functional maps in axial view of the repair network contrast (illusion > natural), superimposed on the normalized MNI brain of one subject. The repair regions included bilateral insula (with MNI center of activity at x = −34 y = 20 z = 0, p_(FDR) = 0.033 for the left hemisphere and x = 38 y = 22 z = 0, p_(FDR) = 0.007 for the right hemisphere), left IFG (Pars opercularis, Brodmann area 44; x = −52 y = 16 z = 4, p_(FDR) = 0.033) and Pre-SMA (x = −6 y = 18 z = 58, p_(FDR) = 0.032). B. Functional maps in axial view of the continuity network contrast (illusion > illusion-failure) superimposed on the normalized MNI brain of one subject. The continuity regions included left AG/STS (x = −42 y = −82 z = 34, p_(FDR) = 0.042), Precuneus (x = −14 y = −52 z = 16, p_(FDR) = 0.042), right STS (x = 50 y = −56 z = 24, p_(FDR) = 0.042), and bilateral SFS (left: x = −22 y = 28 z = 46, p_(FDR) = 0.042; right: x = 28 y = 32 z = 56, p_(FDR) = 0.042). (Region abbreviations: AG= angular gyrus; STS = superior temporal sulcus; IFG = inferior frontal gyrus; SFS = superior frontal sulcus; pre-SMA = pre-supplementary motor area).

Fig. 5. Word-level template-matching in illusory filling-in

The results of the ANOVAs (3 conditions × 2 tasks) contrasting the BOLD signal intensity for the repair (A) and continuity (B) regions (depicted in figure 4). The p values depict the interaction probability. A significant interaction means that repair or continuity mechanisms distinguish between words and pseudowords. This implies reliance on word-level template matching for filling-in missing sensory information. Regions with significant interactions are denoted with asterisks. Two-tailed standard errors are depicted by the error bars. (Region abbreviations: AG= angular gyrus; STS = superior temporal sulcus; IFG = inferior frontal gyrus; pre-SMA = pre-supplementary motor area; SFS = superior frontal sulcus).

Fig. 6. Perceptual sensitivity in early auditory cortex

Top. Functional map depicting activity in the entire right Heschl’s gyrus (orange) and middle right Heschl’s gyrus (dark brown). Bottom left. Average BOLD intensity (compared to baseline) across the entire right Heschl’s gyrus for the three conditions of the word task. Significance was examined using T-test analysis of the average BOLD intensity between conditions. Bars above columns depict standard deviation. Bottom right. Average BOLD intensity (compared to baseline) in middle right Heschl’s gyrus for the three conditions of the word task. Contrasts with asterisks indicate significant differences for all voxels in the region, based on brain-wide t-test corrected at p (FDR) < 0.05. The middle right Heschl’s gyrus shows significant differences between the illusion-failure and the other two conditions, but not between the illusion and natural conditions. Activity at the entire left Heschl’s gyrus (not shown) were also analyzed and yielded similar results as for the entire right Heschl’s gyrus.

Fig. 7. Functional connectivity

Functional connectivity between the left AG/STS seed (continuity region: green-red gradient) with all repair areas (blue-green gradient). The functional connectivity test was conducted on the repair contrast (illusion > natural). This shows regions with which AG communicates more when unconscious repair succeeds in creating a continuous percept. (Region abbreviations: AG= angular gyrus; STS = superior temporal sulcus; IFG = inferior frontal gyrus; pre-SMA = pre-supplementary motor area).

Fig. 8. Model

A schematic of the processing paths mediating perceptual filling-in of speech. Regions with names in orange show evidence for word-level template matching. (Region abbreviations: AG= angular gyrus; STS = superior temporal sulcus; IFG = inferior frontal gyrus; SFS = superior frontal sulcus; pre-SMA = pre-supplementary motor area).