The tracking of speech envelope in the human cortex

Abstract

Humans are highly adept at processing speech. Recently, it has been shown that slow temporal information in speech (i.e., the envelope of speech) is critical for speech comprehension. Furthermore, it has been found that evoked electric potentials in human cortex are correlated with the speech envelope. However, it has been unclear whether this essential linguistic feature is encoded differentially in specific regions, or whether it is represented throughout the auditory system. To answer this question, we recorded neural data with high temporal resolution directly from the cortex while human subjects listened to a spoken story. We found that the gamma activity in human auditory cortex robustly tracks the speech envelope. The effect is so marked that it is observed during a single presentation of the spoken story to each subject. The effect is stronger in regions situated relatively early in the auditory pathway (belt areas) compared to other regions involved in speech processing, including the superior temporal gyrus (STG) and the posterior inferior frontal gyrus (Broca's region). To further distinguish whether speech envelope is encoded in the auditory system as a phonological (speech-related), or instead as a more general acoustic feature, we also probed the auditory system with a melodic stimulus. We found that belt areas track melody envelope weakly, and as the only region considered. Together, our data provide the first direct electrophysiological evidence that the envelope of speech is robustly tracked in non-primary auditory cortex (belt areas in particular), and suggest that the considered higher-order regions (STG and Broca's region) partake in a more abstract linguistic analysis.

Figure 1. High gamma activity in human auditory cortex tracks the envelope of speech.

Black: Time course of the speech envelope. Green: High gamma activity recorded by a channel positioned in the belt areas in subject C (see Figure 5) while the subject listened to a narrated story. For the visualization purpose of this figure, we graphically scaled the magnitude of the neural signal to the magnitude of the envelope signal. The Spearman correlation between the two signals is .

Figure 2. Tracking of speech envelope for each channel located in the belt areas.

The Spearman correlation coefficient () between the speech envelope and gamma activity for each channel in belt areas surrounding the auditory core. Filled bars denote the cases of significant correlation (). The star refers to the example channel shown in Figure 1 and Figure 5.

Figure 3. High gamma activity tracks the envelope of speech better than does the raw potential.

MeanSEM Spearman correlation coefficient () between each neural signal and speech envelope. The mean is computed across all channels in the belt areas ().

Figure 4. Tracking of speech envelope in three auditory cortical regions.

MeanSEM Spearman correlation coefficient () between neural activity and speech envelope in each region of interest. The mean is computed over all channels in each area (, , ). Stars denote the significance of the difference in means (two-tailed t-test), *, **.

Figure 5. Neural tracking of speech envelope at each recording site in each subject.

Color hue (see colorbars) gives at each channel for the individual subjects (A–E), and for the subject average (AVG). Individual channels implanted in each subject are shown in green (belt areas), orange (STG), red (Broca's region), or black (other regions). The location of each channel was determined using the Talairach Atlas daemon (see Methods). In subject C, the arrow points to the channel for which we illustrated the tracking effect (Figure 1). STS: superior temporal sulcus; SF: Sylvian fissure; TTS: transverse temporal sulcus (perpendicular to the view plane).

Figure 6. Tracking of speech envelope in early auditory regions compared to all other regions.

MeanSEM Spearman correlation coefficient () between neural activity and speech envelope in belt areas (green) and all other regions (gray). The mean is computed over all channels in each case (, ).

Figure 7. Neural tracking of speech envelope at each frequency of the sound.

MeanSEM Spearman correlation coefficient () between neural activity and speech envelope for each frequency, for each region of interest. The dashed line gives the average spectrum of the speech in logarithmic units.

Figure 8. Tracking of speech-related and melodic stimuli in human cortex.

MeanSEM Spearman correlation coefficient () between neural activity and the speech envelope, for each region of interest. Left: stimulus containing lyrics. Right: melody. The mean is computed across all channels in each area. Stars denote the significance of the difference in means (t-test), *, **.