Chinese-English bilinguals show linguistic-perceptual links in the brain associating short spoken phrases with corresponding real-world natural action sounds by semantic category

Abstract

Higher cognitive functions such as linguistic comprehension must ultimately relate to perceptual systems in the brain, though how and why this forms remains unclear. Different brain networks that mediate perception when hearing real-world natural sounds has recently been proposed to respect a taxonomic model of acoustic-semantic categories. Using functional magnetic resonance imaging (fMRI) with Chinese/English bilingual listeners, the present study explored whether reception of short spoken phrases, in both Chinese (Mandarin) and English, describing corresponding sound-producing events would engage overlapping brain regions at a semantic category level. The results revealed a double-dissociation of cortical regions that were preferential for representing knowledge of human versus environmental action events, whether conveyed through natural sounds or the corresponding spoken phrases depicted by either language. These findings of cortical hubs exhibiting linguistic-perceptual knowledge links at a semantic category level should help to advance neurocomputational models of the neurodevelopment of language systems.

Keywords: grounded cognition; hearing perception; natural language processing (Alternatives: second language, glottogenesis, fMRI); oral communication.

Figure 1.

A neurobiological taxonomic model of acoustic-semantic categories of natural sounds that are processed as meaningful events along different neuronal pathways in the human brain, based largely on hemodynamic neuroimaging (from Brefczynski-Lewis and Lewis, 2017), reprinted with permission by the publishers.

Figure 2.

(A) Features from the neurobiological taxonomic model (from Fig. 1) that were being tested in the present study, contrasting brain responses to action sounds (green box) relative to speech (red text), and further contrasting the semantic category human action sounds (yellow box) versus environmental action sounds (blue box). Resulting cortical activation patterns of Chinese/English bilingual listeners (n=16) projected onto (B) fiducial and (C) inflated 3-dimensional surface models of the brain (standardized PALS atlas). Colored cortices depict regions differentially responsive for representing knowledge of human actions (yellow hues, p_uncorrected < 0.0005; p_corrected < 0.05) versus knowledge of environmental action events (blue hues), whether heard and correctly categorized as natural sound stimuli (perceptual system, PS), as corresponding spoken Chinese phrases (Chi), or corresponding spoken English phrases (Eng). Regions preferential for processing language (transparent red) versus the natural sound stimuli (transparent green) are also illustrated for comparison (both at p_uncorrected < 0.0005; p_corrected < 0.05). Histograms show the BOLD percent signal change (average ± SEM) for various regions of interest illustrating activation preference for the human (H) versus environmental (E) category of action sound, whether heard as natural sounds or spoken phrases in English or Chinese. IPL = intraparietal lobule; MFG = middle frontal gyrus; SFG = superior frontal gyrus; STG = superior temporal gyrus; TPO-J = temporal-parietal-occipital junction. Refer to text for other details.

Figure 3.

Assessment of the language dominant hemisphere across participants. Regions of interest (ROIs) were derived to quantitatively analyze the degree of lateralized activation in response to the English and Chinese spoken phrase stimuli in the n=16 right handed participants. Estimated location of (A) “Broca’s area” and (B) “Wernicke’s area” and their mirror opposite right hemisphere ROIs. Histograms reveal activation magnitudes in each ROI both as group averages and plotted for each individual. Refer to text for other details.