From words to phrases: neural basis of social event semantic composition

Abstract

Events are typically composed of at least actions and entities. Both actions and entities have been shown to be represented by neural structures respecting domain organizations in the brain, including those of social/animate (face and body; person-directed action) versus inanimate (man-made object or tool; object-directed action) concepts. It is unclear whether the brain combines actions and entities into events in a (relative) domain-specific fashion or via domain-general mechanisms in regions that have been shown to support semantic and syntactic composition. We tested these hypotheses in a functional magnetic resonance imaging experiment where two domains of verb-noun event phrases (social-person versus manipulation-artifact, e.g., "hug mother" versus "fold napkin") and their component words were contrasted. We found a series of brain region supporting social-composition effects more strongly than the manipulation phrase composition-the bilateral inferior occipital gyrus (IOG), inferior temporal gyrus (ITG) and anterior temporal lobe (ATL)-which either showed stronger activation strength tested by univariate contrast, stronger content representation tested by representation similarity analysis, or stronger relationship between the neural activation patterns of phrases and synthesis (additive and multiplication) of the neural activity patterns of the word constituents. No regions were observed showing evidence of phrase composition for both domains or stronger effects of manipulation phrases. These findings highlight the roles of the visual cortex and ATL in social event compositions, suggesting a domain-preferring, rather than domain-general, mechanisms of verbal event composition.

Keywords: Phrase comprehension; Semantic composition; Semantic domain organization; Social event; fMRI.

Fig. 1

Experimental design. a Procedures in a run. Each trial consisted of 3 s stimuli presentation and a 1 to 13 s jitter fixation. b Spatial location of the bilateral ATL and bilateral AG, which were defined by the Harvard–Oxford Atlas. c Hypotheses testing of domain-general and domain-preference composition effects across three analyses

Fig. 2

Whole-brain univariate analysis results. a Whole-brain univariate analysis results contrasting social person meaningfully matched phrases (e.g., “hug mother”) and manipulation artifact meaningfully matched phrases (e.g., “fold napkin”). b Whole-brain univariate analysis results contrasting single words in the social condition (social actions and persons) versus single words in the manipulation condition (manipulation actions and artifacts). Both thresholds were set at voxel level p < 0.001, cluster-extent FWE corrected p < 0.05

Fig. 3

RSA results. a Theoretical RDM of social- and manipulation-matched phrases. b Theoretical RDM of single words in the social and manipulation conditions. c ROI-level RSA results for phrases. d ROI-level RSA results for single words. The * above bars indicate a significant difference from zero (two-tailed, p < 0.05). The four anatomical ROIs were the bilateral ATL and AG, which have been shown in the literature to support semantic and syntactic composition; two functional ROIs (bilateral IOG) were those showed domain preference to social-action-person phrases (see Fig. 2a). e Whole-brain RSA results of socially matched phrases. Brain regions showing significant positive correlations with socially matched RDMs (threshold: voxel level p < 0.001 and cluster extent FWE p < 0.05). No positive results were found for manipulation-matched phrases in these analyses

Fig. 4

Results of phrase-and-synthesizing-components pattern correlation analyses. a A flowchart of phrase-and-synthesizing-components pattern correlation analyses, i.e., correlation between actual activation patterns observed in response to phrases and multiplicative or additive synthetic patterns observed in response to its corresponding component words. b Correlation between phrase activity patterns and synthetic-sum patterns observed in response to word components in the anatomical ROIs and functional ROIs. c Correlation between phrase activity patterns and synthetic-product patterns observed in response to word components in the anatomical ROIs and functional ROIs. Socially matched R means correlations between the observed activation patterns of social-action-person phrases and the synthesis of the activation patterns of the corresponding component words; manipulation-matched R means correlations between the observed activation patterns of manipulation-artifact phrases and the synthesis of the activation patterns of the corresponding component words. Unmatched-R means correlations between the observed activation patterns of the unmatched phrases and the synthesis of the activation patterns of their corresponding component words. The * above bars indicate a significant difference from zero (two-tailed, p < 0.05), and the * above black lines indicate a significant difference between the two conditions (two-tailed, p < 0.05). d Contrast between whole-brain maps of socially matched R and manipulation-matched R for the additive model (threshold: voxel level p < 0.001 and cluster extent FWE p < 0.05). e Contrast between whole-brain maps of socially matched R and manipulation-matched R for the multiplicative model (threshold: voxel level p < 0.001 and cluster extent FWE p < 0.05)