A common framework for semantic memory and semantic composition

Abstract

How the brain constructs meaning from individual words and phrases is a fundamental question for research in semantic cognition, language, and their disorders. These two aspects of meaning are traditionally studied separately, resulting in two large, multi-method literatures, which we sought to bring together in this study. Not only would this address basic cognitive questions of how semantic cognition operates but also because, despite their distinct focuses, both literatures ascribe a critical role to the anterior temporal lobe (ATL) in each aspect of semantics. Given these considerations, we explored the notion that these systems rely on common underlying computational principles when activating conceptual semantic representations via single words, versus building a coherent semantic representation across sequences of words. The present pre-registered study used magnetoencephalography and electroencephalography to track brain activity in participants reading nouns and adjective-noun phrases, while integrating conceptual variables from both literatures: the concreteness of nouns (e.g., "lettuce" vs. "fiction") and the denotational semantics of adjectives (subsective vs. privative, e.g., "bad" vs. "fake"). Region-of-interest analyses show that bilateral ATLs responded more strongly to phrases at different time points, irrespective of concreteness. Decoding analyses on ATL signals further revealed a time-varying representational format for adjective semantics, whereas representations of noun concreteness were more stable and maintained for around 300 ms. Further, the neural representation of noun concreteness was modulated by the preceding adjectives: decoders learning concreteness signals in single words generalised better to subsective relative to privative phrases. These findings point to a unified ATL function for semantic memory and composition.

Keywords: compositional generalisation; decoding; magnetoencephalography; semantic cognition; semantic composition; semantic memory.

Fig. 1.

Trial structure. Task probes were included for 10% of trials to encourage attention.

Fig. 2.

Regions of interest.

Fig. 3.

Time-resolved analyses of the ATL ROIs. (A, B) Time series of evoked responses in the left and right ATLs, respectively, for words and subsective phrases, aligned to noun onset (0 seconds). Error bars represent ±1 within-subjects SEM (Loftus & Masson, 1994). Cluster extent identified from permutation testing is indicated by yellow shaded areas for significant effects (p < .05) and grey shaded areas for marginally significant effects (p < .1). (C, D, E) Mean response within the cluster time window, plotted separately for each condition. Error bars represent ±1 within-subjects SEM (Loftus & Masson, 1994). Units: Am = Ampere-meter, S = second.

Fig. 4.

Time-resolved analyses of the left ATL ROI. (A) Time series of evoked responses in the left ATL for subsective and privative phrases, aligned to noun onset (0 seconds). Error bars represent ±1 within-subjects SEM (Loftus & Masson, 1994). Permutation-based clusters are indicated by grey shaded areas for marginally significant effects (p < .1). (B) Mean response within the cluster time window, plotted separately for each condition. Error bars represent ±1 within-subjects SEM (Loftus & Masson, 1994). Units: Am = Ampere-meter, S = second.

Fig. 5.

(A, C) Time series of decoding performance (AUC: area under the curve summarising decoding performance) in the left ATL for adjective (A) and noun (C) semantics, aligned to adjective onset at 0 seconds. Error bands represent ±1 SEM. Shaded regions indicate cluster extents corresponding to group-level effects assessed with cluster-based one-sample permutation tests against chance. (B, D) Temporal generalisation matrices (TGM) of adjective (B) and noun (D) semantics, where time 0 seconds is adjective onset and 0.6 seconds is noun onset. Black and grey outlines denote cluster extents corresponding to significant and marginally significant group-level effects assessed with cluster-based one-sample permutation tests against chance.

Fig. 6.

Decoders trained on single word concreteness generalised to subsective but not privative phrases. (A, C) Diagonal decoding performance over time, where time 0 is noun onset. The light and dark grey boxes in the background denote the time window (early vs. late) extents. (B, D) Temporal generalisation matrices, where train time corresponds to single words and test time corresponds to either subsective phrases (B) or privative phrases (D), where time 0 is noun onset. Error bars represent one within-subjects SEM (Loftus & Masson, 1994, 1994). Shaded areas under decoding curves denote clusters corresponding to significant (coloured) and marginally significant (grey) group-level effects assessed with cluster-based one-sample permutation tests against chance. (E) Bar plot showing averaged responses by time window and test type. The light and dark grey boxes in the background correspond to the time window (early vs. late) extents in the time series plots in (A) and (C).