Timing of brain entrainment to the speech envelope during speaking, listening and self-listening

Abstract

⁠This study investigates the dynamics of speech envelope tracking during speech production, listening and self-listening. We use a paradigm in which participants listen to natural speech (Listening), produce natural speech (Speech Production), and listen to the playback of their own speech (Self-Listening), all while their neural activity is recorded with EEG. After time-locking EEG data collection and auditory recording and playback, we used a Gaussian copula mutual information measure to estimate the relationship between information content in the EEG and auditory signals. In the 2-10 Hz frequency range, we identified different latencies for maximal speech envelope tracking during speech production and speech perception. Maximal speech tracking takes place approximately 110 ms after auditory presentation during perception and 25 ms before vocalisation during speech production. These results describe a specific timeline for speech tracking in speakers and listeners in line with the idea of a speech chain and hence, delays in communication.

Keywords: Entrainment; Perception; Production; Self-listening; Speech.

Fig. 1

Experimental design. Conditions present in each block. In the Listening condition, participants (N = 13) listen to speech recordings produced by two additional participants (auditory signal represented in blue). In Speech Production, participants (N = 15) overtly speak about the same topic that they heard spoken about by the other speaker. During Self-Listening, participants listen to their own recorded speech (N = 15). In Resting-State, participants watch a fixation cross. Four blocks corresponding to four different topics are presented. Presentation order of the Resting-State condition is random inside the block. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 2

Mean lagged speech envelope tracking estimated at the scalp-level and grouped by condition. Negative times indicate the audio signal preceding the EEG signal and positive lags indicate that the brain activity follows the auditory signal. Zero lag is when the auditory and EEG signals are synchronous. Each column represents a condition. Panel A contains the channel-time-lag representation of the sample average. Colour scale represents the magnitude of the GCMI value (bits). Panel B contains the mean (blue line) and standard deviation (blue shadowed area) when collapsing across channels. The black dots on the x-axis indicate the lags in which the GCMI values are increased (95% of confidence). Panel C contains the response topography at the latency showing the maximum GCMI value, which is indicated at the bottom. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Latencies corresponding to the maximum speech envelope tracking estimated by condition. Positive time values represent the time elapsed after the auditory stimulus was recorded, and negative values represent times before the start of the auditory recording. Center lines show the medians over participants; box limits indicate the 25th and 75th percentiles as determined by BoxPlotR (Spitzer, Wildenhain, Rappsilber, & Tyers, 2014); whiskers extend 1.5 times the interquartile range from the 25th and 75th percentiles, outliers are represented by dots; crosses represent sample means; bars indicate 95% confidence intervals of the means. Participant counts: n = 15, 15 and 13 sample points for Self-Listening, Speech Production and Listening, respectively.

Fig. 4

Dipole density plot of those ICs showing the maximum speech envelope tracking, grouped by condition. The colour scale represents the dipole density values, which are normalised such that the voxels of the brain sum to one.