Sensorimotor Representation of Speech Perception. Cross-Decoding of Place of Articulation Features during Selective Attention to Syllables in 7T fMRI

Abstract

Sensorimotor integration, the translation between acoustic signals and motoric programs, may constitute a crucial mechanism for speech. During speech perception, the acoustic-motoric translations include the recruitment of cortical areas for the representation of speech articulatory features, such as place of articulation. Selective attention can shape the processing and performance of speech perception tasks. Whether and where sensorimotor integration takes place during attentive speech perception remains to be explored. Here, we investigate articulatory feature representations of spoken consonant-vowel (CV) syllables during two distinct tasks. Fourteen healthy humans attended to either the vowel or the consonant within a syllable in separate delayed-match-to-sample tasks. Single-trial fMRI blood oxygenation level-dependent (BOLD) responses from perception periods were analyzed using multivariate pattern classification and a searchlight approach to reveal neural activation patterns sensitive to the processing of place of articulation (i.e., bilabial/labiodental vs. alveolar). To isolate place of articulation representation from acoustic covariation, we applied a cross-decoding (generalization) procedure across distinct features of manner of articulation (i.e., stop, fricative, and nasal). We found evidence for the representation of place of articulation across tasks and in both tasks separately: for attention to vowels, generalization maps included bilateral clusters of superior and posterior temporal, insular, and frontal regions; for attention to consonants, generalization maps encompassed clusters in temporoparietal, insular, and frontal regions within the right hemisphere only. Our results specify the cortical representation of place of articulation features generalized across manner of articulation during attentive syllable perception, thus supporting sensorimotor integration during attentive speech perception and demonstrating the value of generalization.

Keywords: 7T fMRI; MVPA-based cross-decoding; place of articulation; selective attention; sensorimotor; speech perception.

Figure 1.

Stimuli and experimental paradigm. A, Spoken stimuli matrix and articulatory properties. The 18 syllables were selected according to the place of articulation (i.e., bilabial/labiodental and alveolar) and manner of articulation (i.e., stop, fricative, and nasal). B, Experimental procedure and task description. Example of a typical match-to-sample trial during attend to vowels task. Subjects received instructions per block, in which they attended to consonants or vowels, respectively, and conducted a match-to-sample decision within slow-event related trials. Each block started with the visual presentation of a task cue (i.e., attention target vowel or consonant), indicating which task to perform in the next 27 trials. Each trial started with a fixation period in which a fixation cross was presented at the center of screen, together with a syllable sound (340 ms). After a jittered ISI (jitter: 6–8 s) a visual cue (i.e., a written letter, vowel, or consonant) was presented for 2 s, followed by the immediate subject’s response by pressing a button either with the right index finger (for match trials) or middle index finger (for mismatch trials). The response was followed by a jittered ISI (4–10 s) to complete the jittered ITI period (12–16 s) before the next trial started. C, Schematic representation of the fMRI acquisition sequence and its relationship to the syllable sounds presented to the participants. Vow, attentd to vowels task; Con, attend to consonants task; Syll, syllable; TA, time of acquisition; TR, time for repetition; Phase, opposite phase encoding volumes acquired for distortion correction.

Figure 2.

Brain areas that show speech sound processing during attend to vowels and attend to consonants tasks. Functional maps depicting the overall pattern of sound-evoked cortical responses during performance of (A) attend to vowels (i.e., all sounds attend to vowels > baseline) and (B) attend to consonants task (i.e., all sounds attend to consonants > baseline); p value at cluster threshold (p-clust) < 0.05. Maps are visualized on inflated and aligned group-averaged representations of the left (LH) and right (RH) hemispheres of the fourteen subjects (light gray, gyri; dark gray, sulci). Color scale indicates average β values.

Figure 3.

Cortical representation of place of articulation features, generalized across manner of articulation. Generalization maps depicting classification accuracies during (A) attend to vowels and (B) attend to consonants tasks. Insula cluster represents the outline (i.e., while line) of the largest continuous uncorrected cluster from the differences between the two tasks; p value at cluster-size threshold (p_clust) < 0.05. Searchlight radius 8.25 mm. Maps are visualized on inflated and aligned group-averaged representations of the left (LH) and right (RH) hemispheres of the fourteen subjects (light gray, gyri; dark gray, sulci). Color scale indicates classification accuracy (Acc) percentages.

Figure 4.

Differences in classification of place of articulation features, generalized across manner of articulation. Maps depicting the difference in the classification accuracies between attend to vowels and attend to consonants tasks. The largest continuous cluster (i.e., area, 134 mm²) was found in the left anterior insula, outlined in white; p value uncorrected (p_uncorr) < 0.05. Searchlight radius 8.25 mm. Maps are visualized on inflated and aligned group-averaged representations of the left (LH) and right (RH) hemispheres of the fourteen subjects (light gray, gyri; dark gray, sulci). Yellow-orange clusters show larger classification accuracies during the attend to vowels task, and green-blue clusters show lager classification accuracies during attend to consonants task. Please note that the color scale does not directly relates to the colors used in Figure 3. Color scale indicates differences in the classification accuracy (Acc) percentages.

Figure 5.

Cortical representation of place of articulation features, generalized across manner of articulation, pooling all trails from two tasks. Generalization maps depicting classification accuracies calculated by pooling all trails from attend to vowels and attend to consonants tasks. Corrected for multiple comparison with cluster-size thresholding. Insula cluster represents the outline (i.e., white line) of the largest continuous uncorrected cluster from the differences between the two tasks; p value at cluster-size threshold (p_clust) < 0.05. Searchlight radius 8.25 mm. Maps are visualized on inflated and aligned group-averaged representations of the left (LH) and right (RH) hemispheres of the fourteen subjects (light gray, gyri; dark gray, sulci). Color scale indicates classification accuracy (Acc) percentages.