Beyond the sentence given

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to Fregean compositionality, which states that the meaning of an utterance is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. Since the domain of syntactic rules is the sentence, the implication of this idea is that language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step, the sentence meaning is integrated with information from prior discourse, world knowledge, information about the speaker and semantic information from extra-linguistic domains such as co-speech gestures or the visual world. Here, we present results from recordings of event-related brain potentials that are inconsistent with this classical two-step model of language interpretation. Our data support a one-step model in which knowledge about the context and the world, concomitant information from other modalities, and the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. Underlying the one-step model is the immediacy assumption, according to which all available information will immediately be used to co-determine the interpretation of the speaker's message. Functional magnetic resonance imaging data that we collected indicate that Broca's area plays an important role in semantic unification. Language comprehension involves the rapid incorporation of information in a 'single unification space', coming from a broader range of cognitive domains than presupposed in the standard two-step model of interpretation.

Figure 1

N400 effects triggered by (a) discourse-related and (b) sentence-related anomalies. Waveforms are presented for a representative electrode site (Pz). The latencies of the N400 effect in discourse and sentence contexts (both onset and peak latencies) are the same (after Van Berkum et al. 2003).

Figure 2

N400 effects triggered by a correct predicate (salted) that is, however, contextually disfavoured in comparison to an incorrect predicate (in love). Waveforms are presented for representative electrode sites, time locked to the onset of the critical inanimate/animate predicate in the fifth sentence (after Nieuwland & Van Berkum 2006).

Figure 3

N400 effects triggered by a critical word (in bold) that rendered the spoken sentence inconsistent with voice-based inferences about the speaker. Three representative electrode sites are shown (speaker-inconsistent waveforms are in red) as well as the topographic distribution of the N400 effect (Van Berkum et al. submitted).

Figure 4

(a) Grand average ERPs for a representative electrode site (Cz) for correct condition (black line), world knowledge violation (green dotted line) and semantic violation (red dashed line). ERPs are time locked to the presentation of the critical words (in italic). Spline-interpolated isovoltage maps display the topographic distributions of the mean differences from 300 to 550 ms between semantic violation and control (left); and between world knowledge violation and control (right). Topographic distributions of the N400 effect are not significantly different between semantic and world knowledge violation (p=0.9). (b) The common activation for semantic and world knowledge violations compared with the correct condition based on the results of a minimum-T-field conjunction analysis. Both violations resulted in a single common activation (p=0.043, corrected) in the LIFG (in, or in the vicinity of, Brodmann's area 45 ([x, y, z]=[−44, 30, 8]; Z=4.87) and brain area (BA 47) ([x, y, z]=[−48, 28, −12]; Z=4.15). The cross hair indicates the voxel of maximal activation and has the following coordinates [x, y, z]=[−44, 30, 8] (left BA 45).

Figure 5

Grand-average waveforms for ERPs elicited in the three semantic mismatch conditions and the correct condition at two representative electrode sites (FC1 and FC2). Negativity is plotted upwards. Waveforms are time locked to the onset of spoken verb and gesture (0 ms).

Figure 6

Gesture and speech in a sentence context. Mean activation levels (β weights) for the four experimental conditions in left inferior frontal cortex (BA 45/47). The activation levels are averaged over participants. An asterisk indicates a significant difference of the activation level of that condition compared with the correct condition (G+L+), at an α level of p<0.05. Error bars are standard error of the mean. G+L+, correct condition; G+L−, language mismatch; G−L+, gesture mismatch; G−L−, double mismatch.