Attentional Modulation of the Cortical Contribution to the Frequency-Following Response Evoked by Continuous Speech

Abstract

Selective attention to one of several competing speakers is required for comprehending a target speaker among other voices and for successful communication with them. It moreover has been found to involve the neural tracking of low-frequency speech rhythms in the auditory cortex. Effects of selective attention have also been found in subcortical neural activities, in particular regarding the frequency-following response related to the fundamental frequency of speech (speech-FFR). Recent investigations have, however, shown that the speech-FFR contains cortical contributions as well. It remains unclear whether these are also modulated by selective attention. Here we used magnetoencephalography to assess the attentional modulation of the cortical contributions to the speech-FFR. We presented both male and female participants with two competing speech signals and analyzed the cortical responses during attentional switching between the two speakers. Our findings revealed robust attentional modulation of the cortical contribution to the speech-FFR: the neural responses were higher when the speaker was attended than when they were ignored. We also found that, regardless of attention, a voice with a lower fundamental frequency elicited a larger cortical contribution to the speech-FFR than a voice with a higher fundamental frequency. Our results show that the attentional modulation of the speech-FFR does not only occur subcortically but extends to the auditory cortex as well.SIGNIFICANCE STATEMENT Understanding speech in noise requires attention to a target speaker. One of the speech features that a listener can use to identify a target voice among others and attend it is the fundamental frequency, together with its higher harmonics. The fundamental frequency arises from the opening and closing of the vocal folds and is tracked by high-frequency neural activity in the auditory brainstem and in the cortex. Previous investigations showed that the subcortical neural tracking is modulated by selective attention. Here we show that attention affects the cortical tracking of the fundamental frequency as well: it is stronger when a particular voice is attended than when it is ignored.

Keywords: cortical response; selective attention; speech processing; speech-FFR.

Figure 1.

Experimental setup and acoustic stimuli. a, Two audiobooks (one attended and one ignored) were presented simultaneously while MEG was recorded. b, One of the two male speakers had a lower fundamental frequency and hence pitch (LP), and the other speaker a higher one (HP). c, We quantified the speech-FFR through two acoustic features: the fundamental waveform that reflected the portion of the speech spectrogram around the fundamental frequency, and the envelope modulation of the higher harmonics.

Figure 2.

Cortical responses to both acoustic features of the LP and the HP voice. The voxel-averaged and subject-averaged envelopes of the TRF amplitudes are significant at time lags of ∼34 ms (dashed gray lines) for both features and both in the LP-A condition (LP-A env., red) and in the LP-I condition (LP-I env., black). The TRF magnitudes display oscillation at $2f_0$ and are displayed as well (LP-A mag., pink; LP-I mag., gray). Significant time delays are indicated through the colored bars at the bottom of the plots. The corresponding brain plots show right-lateralized activity at 34 ms (attended, top row; ignored, bottom row). The same applies for the response to the fundamental waveform in the HP-A and HP-I conditions. The envelope modulation for the HP voice shows significant activity at time lags of ∼30 ms (dashed gray line) when the speaker was attended, but not when he was ignored.

Figure 3.

Comparison of the responses to the LP voice to the responses to the HP speaker. We show the peak of the envelope of the TRF magnitudes. The responses to the fundamental waveform (top left) and to the envelope modulation (top right) were significantly higher for the LP voice than for the HP voice when these voices were attended (two-tailed Wilcoxon signed-rank test). When the voices were ignored, the difference between the responses to the LP voice and the HP voice were significant for the envelope modulation (bottom left), but not for the fundamental waveform (bottom right, two-tailed Wilcoxon signed-rank test). The error bars indicate the SEM when averaging across subjects.

Figure 4.

Responses to the fundamental waveform of the LP speaker. a, Attentional modulation of the cortical contribution to the speech-FFR. For 17 of 22 subjects, the peak in the envelope of the TRF magnitudes at a delay of 34 ms showed a significant difference between the attended (red) and the ignored (black) condition ($\mathrm{*}$, $0.01\le p<0.05$; $\mathrm{**}$, $0.001\le p<0.01$; $\mathrm{***}$, $p<0.001$). The same behavior emerged regarding the population-average response (avg.). b, Cortical TRFs and corresponding voxel magnitudes for the LP-A condition (LP-A env., red; LP-A mag., pink; top brainplots) and the LP-I condition (LP-I env., black; LP-I mag., gray; bottom brainplots) speaker for the exemplary subject 9 (left) and subject 3 (right). There is a large effect of attention for subject 9 and a much smaller one for subject 3.

Figure 5.

Responses to the envelope modulation for the LP speaker. a, Attentional modulation of the cortical contribution to the speech-FFR. For 14 of 22 subjects, the envelope of the TRF magnitude at a delay of 34 ms (peak of the envelope) showed a significant difference between the attended condition (red) and the ignored condition (black; $\mathrm{*}$, $0.01\le p<0.05$; $\mathrm{**}$, $0.001\le p<0.01$; $\mathrm{***}$, $p<0.001$). The population-average TRF (avg.) shows the same attentional modulation. b, The cortical TRFs and the corresponding voxel magnitudes for the LP-A condition (LP-A env., red; LP-A mag., pink; top brainplots) and the LP-I condition (LP-I env., black; LP-I mag., gray; bottom brainplots) for exemplary subject 9 (left) and subject 3 (right). The channel-averaged TRFs for subject 9 show a strong attentional modulation, while the one for subject 3 is insignificant.

Figure 6.

Cortical responses to the fundamental waveform of the HP speaker. a, Attentional modulation of the cortical response. For 7 of 22 subjects, the peak envelope of the TRF magnitude, at a delay of 34 ms, showed a significant difference between the attended (red) and the ignored (gray) conditions ($\mathrm{*}$, $0.01\le p<0.05$; $\mathrm{**}$, $0.001\le p<0.01$; $\mathrm{***}$, $p<0.001$). The population-average TRF (avg.) displayed the attentional effect as well. b, The time course of the TRF magnitudes and the corresponding envelopes as well as the corresponding voxel magnitudes for the HP-A condition (HP-A env., red; HP-A mag., pink; top brainplot) and the HP-I condition (HP-I env., black; HP-I mag., gray; bottom brainplots) for exemplary subject 22 (left) and subject 7 (right). The cortical response of subject 22 showed a significant effect of selective attention, but not the response of subject 7.

Figure 7.

Cortical responses to the envelope modulation of the HP speaker. a, Attentional modulation of the cortical contribution to the speech-FFR. For 6 of 22 subjects, the peak envelope of the TRF magnitude, at a delay of 30 ms, differed significantly between the attended (red) condition and the ignored (black) condition ($\mathrm{*}$, $0.01\le p<0.05$; $\mathrm{**}$, $0.001\le p<0.01$; $\mathrm{***}$, $p<0.001$). The population average (avg.) showed the attentional modulation as well. b, Envelopes of the TRF magnitudes as well as the TRF magnitudes themselves and the corresponding voxel magnitudes for the HP-A condition (HP-A env., red; HP-A mag., pink; top brainplot) and the HP-I condition (HP-I env., black; HP-I mag., gray; bottom brainplot) for exemplary subjects 22 (left) and subject 7 (right). The cortical response of subject 22 showed a significant attentional effect, whereas that of subject 7 did not.

Figure 8.

Correlation analysis of attention and participant behavior. No significant correlation emerged between the correct answer score of the different participants and their attentional modulation index I, either for the fundamental waveform feature (left) or for the envelope modulation feature (right). Significant correlations were absent both for the LP speaker (blue dots) as well as for the HP speaker (brown squares). The correlation was investigated by calculating Pearson's correlation coefficient r as well as the according p-value.