Functional organization of human sensorimotor cortex for speech articulation

Abstract

Speaking is one of the most complex actions that we perform, but nearly all of us learn to do it effortlessly. Production of fluent speech requires the precise, coordinated movement of multiple articulators (for example, the lips, jaw, tongue and larynx) over rapid time scales. Here we used high-resolution, multi-electrode cortical recordings during the production of consonant-vowel syllables to determine the organization of speech sensorimotor cortex in humans. We found speech-articulator representations that are arranged somatotopically on ventral pre- and post-central gyri, and that partially overlap at individual electrodes. These representations were coordinated temporally as sequences during syllable production. Spatial patterns of cortical activity showed an emergent, population-level representation, which was organized by phonetic features. Over tens of milliseconds, the spatial patterns transitioned between distinct representations for different consonants and vowels. These results reveal the dynamic organization of speech sensorimotor cortex during the generation of multi-articulator movements that underlies our ability to speak.

Figure 1. vSMC Physiology During Syllable Production

a, MRI reconstruction of single subject brain with vSMC electrodes (dots), colored by distance from Sylvian fissure. b, Expanded view of vSMC anatomy: pre- and post-central gyri (PrCG and PoCG), central sulcus (cs), Sylvian fissure (Sf). Scale bar =1 cm. c–e (top), Vocal tract schematics for three consonants (/b/, /d/, /g/), produced by occlusion at the lips, tongue tip, and tongue body, respectively (red arrow). (middle) Spectrograms of spoken consonant-vowel (CV) syllables. (bottom) Average cortical activity from subset of electrodes. Vertical, dashed line is acoustic onset of CV transition. f–h, Cortical activity at selected electrodes for different phonetic contrasts (mean ± s.e.). Acoustic waveforms displayed above. f, Fricatives [/θ/(“th” of “thin”), /s/, /ʃ/(“sh” of “shin”)] with different constriction locations. g, Front tongue consonants (/l/, /n/, /d/) with different constriction degree/shapes. h, Single consonant [/j/ (“y” of “yes”)] with different vowels (/a/, /i/, /u/). Red arrow corresponds to a tongue electrode with prolonged activity for /i/ and /u/ vowels. Black arrow corresponds to active lip electrode for /u/.

Figure 2. Spatial Representation of Articulators

a, Localization of lips, jaw, tongue, and larynx representations. Average magnitude of articulator weightings (color scale) plotted as a function of anterior-posterior (AP) distance from the central sulcus and dorsal-ventral (DV) distance from the Sylvian fissure (n = 3 subjects). b, Functional somatotopic organization of speech articulator representations in vSMC. Lips (L, red); jaw (J, green); tongue (T, blue); larynx (X, black), mixed (Gold). Letters correspond to locations based upon direct measurement-derived regression weights, shaded rectangles correspond to regions classified by k-nearest neighbor.

Figure 3. Temporal Representation of Articulators

a–b, Timing of correlations between cortical activity and consonant (a) and vowel (b) articulator features. Heat maps display correlation coefficients (R) for a subset of electrodes. c, Acoustic landmarks. Onset (<), peak power (o) and offset (>) for CV syllables (mean ± s.e., n = 168 syllables, all subjects). (x) is vowel midpoint. s.e. bars are smaller than symbols. d, Temporal sequence and range of correlations. Symbols same as in (c). Data are mean (symbol) ± s.e. (solid line) across electrodes from all subjects. Number of electrodes contributing to each articulator is displayed on the right.

Figure 4. Phonetic Organization of Spatial Patterns

a–b, Scatterplots of CV syllables in the first three principal components for consonants (t = −25 ms) (a) and vowels (t = +250 ms) (b) A subset of CVs are labeled, all others have dots. Coloring denotes k-means cluster membership. c–d, Hierarchical clustering of cortical state-space at consonant and vowel time points. Individual syllables are color-coded and dendrogram branches are labeled by known linguistic categories. e–f, Correlations between cortical state-space and phonetic features. Black line: median; grey box: 25^th and 75^th percentile. ***: P < 10⁻¹⁰, WSRT; n = 297 for both consonants and vowels.

Figure 5. Dynamics of Phonetic Representations

a–b, Cortical state-space trajectories. a, Consonants transitioning to the vowel /u/ (red-sibilant, green-coronal tongue, blue-dorsal tongue, black-labial). Each line corresponds to a single CV trajectory. Symbols; left triangle: t = −500ms, square: t = −25ms, circle: t = 250ms, right triangle: t = 750ms. b, Trajectories of the labial consonants transitioning to /a/, /i/ and /u/(cyan, magenta, and yellow, respectively). c–d Across-subject averages of cluster separability (c) and correlation between cortical state-space structure and phonetic features (d) for consonants (red) and vowels (black) (mean ± s.e). e, Time-course of CV syllable trajectories for Subject 1. Each color corresponds to one of the consonant or vowel groups.