Processing of auditory feedback in perisylvian and insular cortex

Abstract

When we speak, we not only make movements with our mouth, lips, and tongue, but we also hear the sound of our own voice. Thus, speech production in the brain involves not only controlling the movements we make, but also auditory and sensory feedback. Auditory responses are typically suppressed during speech production compared to perception, but how this manifests across space and time is unclear. Here we recorded intracranial EEG in seventeen pediatric, adolescent, and adult patients with medication-resistant epilepsy who performed a reading/listening task to investigate how other auditory responses are modulated during speech production. We identified onset and sustained responses to speech in bilateral auditory cortex, with a selective suppression of onset responses during speech production. Onset responses provide a temporal landmark during speech perception that is redundant with forward prediction during speech production. Phonological feature tuning in these "onset suppression" electrodes remained stable between perception and production. Notably, the posterior insula responded at sentence onset for both perception and production, suggesting a role in multisensory integration during feedback control.

Keywords: auditory perception; intracranial electrophysiology; language; speech; speech motor control; speech production.

Figure 1:. Auditory onset responses are suppressed during speech production.

(A) Schematic of reading and listening task. Participants read a sentence aloud (purple) then passively listened to playback of themselves reading the sentence (green). Pink spikes in the beginning and middle of the audio waveform indicate inter-trial click tones, used as a cue and an auditory control. (B) Single-electrode plots showing different profiles of response selectivity across the cortex. Color gradient represents normalized SI values. A more positive SI indicates an electrode is more responsive to speech perception stimuli (e1) while a more negative SI means an electrode is more responsive to production stimuli (e3). e2 and e3 are examples of response profiles described in subsequent figures (Figures 2 and 3, respectively). Subplot titles reflect the participant ID and electrode name from the clinical montage. (C) Whole-brain and single-electrode visualizations of perception and production selectivity (SI). Electrodes are plotted on a template brain with an inflated cortical surface; dark gray indicates sulci while light gray indicates gyri. Single-electrode plots of high-gamma activity demonstrate suppression of onset response relative to the acoustic onset of the sentence (vertical black line). (D) Box plot of suppression index during onset (blue) and sustained (orange) time windows separated by anatomical region of interest in primary and non-primary auditory cortex. Brackets indicate significance (* = p < 0.05; ** = p < 0.01). Abbreviations: HG: Heschl’s gyrus; PT: planum temporale; STG: superior temporal gyrus; STS: superior temporal sulcus; MTG: middle temporal gyrus; CS: central sulcus; Post. Ins.: posterior insula.

Figure 2:. A functional region of interest in posterior insula shows onset responses to both speaking and listening.

(A) Whole-brain and visualization of dual onset electrodes. Electrodes are plotted on a template brain with an inflated cortical surface; dark gray indicates sulci while light gray indicates gyri. Black outline on template brain highlights functional region of interest in posterior insula with anatomical structures labeled. Electrode color indicates the difference in Z-scored high gamma peaks during the speaking and listening conditions (∆Z). Right hemisphere is cropped to emphasize insula ROI, while left hemisphere is shown in entirety due to lower number of electrodes. (B) Whole-brain visualization of electrodes with onset responses only during speech perception. Electrode color indicates the peak high gamma amplitude during the onset response. (C) Whole-brain visualization of electrodes with onset responses only during speech production. Electrode color indicates the peak high gamma amplitude during the onset response. (D) Single electrode activity from posterior insular electrodes highlighting dual onset responses during speech production and perception. Vertical black line indicates acoustic onset of sentence. Subplot titles reflect the participant ID, electrode name from the clinical montage, and anatomical ROI. (E) Grayscale heatmaps of single-trial electrode activity during a nonspeech motor control task, separated by no vocalization (e.g., “stick your tongue out”) and vocalization (e.g., “say ‘aaaa’”). For vocalization trials, onset of acoustic activity is visualized relative to the click accompanying the presentation of instructions (pink) and the onset of vocalization (red). (F) Strip plot showing the distribution of channel-by-channel onset response peak amplitudes separated by anatomical region of interest and whether onset responses occur only during perception (left), only during production (center), or occur during perception and production (right). Electrodes are colored according to the colormaps of (A), (B), and (C). (G) Schematic of quantification of onset response for an example electrode (e2, DC5 PSF-PI3). The first contiguous peak of activity >1.5 SD above the mean response constitutes the onset response and is shaded in orange. Peak amplitude values displayed in (B), (C) and (G) are indicated. (H) Bar plot showing the estimated marginal mean latency of the onset response in three regions of interest: auditory primary (HG + PT), auditory non-primary (STG + STS), and posterior + inferior insular. Insular onset latency is comparable to primary auditory latency. Brackets indicate significance (* = p < 0.05; ** = p < 0.01). Abbreviations: HG: Heschl’s gyrus; STG: superior temporal gyrus; STS: superior temporal sulcus; MTG: middle temporal gyrus; Inf/Sup/Ant/Post/ CrS: inferior/superior/anterior/posterior circular sulcus of the insula; LGI: long gyrus of the insula; SGI: short gyrus of the insula; PT: planum temporale.

Figure 3.. Anatomically distinct onset suppression and dual onset clusters represent a subclass of response profiles to continuous speech production and perception.

(A) Percent variance explained by cNMF as a function of total number of clusters in factorization. Threshold of k = 9 factorization plotted as vertical black line. (B) cNMF identifies three response profiles of interest: (c1) onset suppression electrodes, characterized by a suppression of onset responses during speech production and localized to STG/HG; (c2) dual onset electrodes, characterized by the presence of onset responses during perception and production and localized to posterior insula; (c3) pre-articulatory motor electrodes, characterized by activity prior to acoustic onset of stimulus during speech production and localized to ventral sensorimotor cortex. Left: Cluster basis functions for speaking sentences (purple), listening to sentences (green), and inter-trial click (pink) for c1, c2, and c3. Center, right: Two example electrodes from the top 16 weighted electrodes. Subplot titles reflect the participant ID and electrode name from the clinical montage. (C) Cropped template brain showing top 50 weighted electrodes for individual clusters (c1, c2, c3). A darker red electrode indicates higher within-cluster weight. (D) Individual electrode contribution to dual onset and onset suppression cNMF clusters in both hemispheres. Top 50 weighted electrodes for each cluster are plotted on a template brain with an inflated cortical surface; dark gray indicates sulci while light gray indicates gyri. Red electrodes contribute more weight to the “onset suppression” cluster while blue electrodes contribute more to the “dual onset” cluster; purple electrodes contribute equally to both clusters while white electrodes contribute to neither. (E) Percent similarity of onset suppression (c1) and dual onset (c2) clusters’ top 50 electrodes. The majority of the electrode weighting across these two clusters is non-overlapping. Abbreviations: STG: superior temporal gyrus; CS: central sulcus. Inf. Ins. = inferior insula, Post. Ins = posterior insula.

Figure 4.. Playback consistency manipulation yields separate, weaker effects than onset suppression.

(A) Task schematic showing playback consistency manipulation. Participants read a sentence aloud (purple) then passively listened to playback of that sentence (blue) or randomly selected playback of a previous trial (orange). (B) Whole-brain visualization of responsiveness to playback consistency. Electrodes are plotted on an inflated template brain; dark gray indicates sulci while light gray indicates gyri. Electrodes are colored using a 2D colormap that represents high gamma amplitude during consistent and inconsistent playback; blue indicates a response during consistent playback but not during inconsistent, orange indicates a response during inconsistent playback but not during consistent playback, pink indicates a response to both playback conditions, white indicates a response to neither. Most electrodes are pink, indicating strong responses to both conditions. Example electrodes from (D) are indicated. (C) Scatter plot of channel-by-channel peak high-gamma activity during consistent playback (Y-axis) and inconsistent playback (X-axis). Vertical black line indicates unity. Color corresponds to gross anatomical region. Example electrodes from (D) are indicated. (D) Single-electrode plots of high-gamma activity relative to sentence onset (vertical black line). Left column (e1 and e2): Electrodes in temporal cortex demonstrating a slight preference for inconsistent playback. Right column (e3 and e4): Electrodes in frontal/parietal cortex demonstrating a slight preference for consistent playback and a larger preference for speech production trials. Abbreviations: HG: Heschl’s gyrus; STG: superior temporal gyrus; PreCS: precentral sulcus; Supramar: supramarginal gyrus.

Figure 5.. Phonological feature tuning is stable during speaking and listening across brain regions.

(A) Regression schematic. Fourteen phonological features corresponding to place of articulation, manner of articulation, and presence of voicing alongside four features encoding task-specific information (i.e., whether a phoneme took place during a speaking or listening trial, the playback condition during the phoneme) were binarized sample-by-sample to form a stimulus matrix for use in temporal receptive field modeling. (B) Model performance as measured by the linear correlation coefficient (r) between the model’s prediction of the held-out sEEG and the actual response plotted at an individual electrode level on an inflated template brain; dark gray indicates sulci while light gray indicates gyri. Example electrodes from (D) and (E) are indicated. (C) Model performance by region of interest. Color corresponds to gross anatomical region. (D) Temporal receptive fields of two example electrodes in temporal and insular cortex. (E) Temporal receptive fields of an example electrode for the four models presented in (F). (F) Scatter plot of channel-by-channel linear correlation coefficients (r) colored by model comparison. The X-axis shows performance for the “base” model whose schematic is presented in (A). The Y-axis for each scatterplot shows performance for a modified version of the base model: task features encoding production and perception were removed from the model (yellow); task features encoding consistent and inconsistent playback conditions were removed from the model (cyan); phonological features were separated into production-specific, perception-specific, and combined spaces (magenta). Abbreviations: HG: Heschl’s gyrus; PT: planum temporale; STG/S: superior temporal gyrus/sulcus; MTG/S: middle temporal gyrus/sulcus; PreCG/S: precentral gyrus/sulcus; CS: central sulcus; SFG/S: superior frontal gyrus/sulcus; MFG/S: middle frontal gyrus/sulcus; IFG/S: inferior frontal gyrus/sulcus; OFC: orbitofrontal cortex; SPL: superior parietal lobule; PostCG: postcentral gyrus; Ant./Post./Sup./Inf. Ins.: anterior/posterior/superior/inferior insula.