A Neurocomputational Model of the N400 and the P600 in Language Processing

Abstract

Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component-traditionally taken to reflect semantic integration-but instead produced a P600 effect, which is generally linked to syntactic processing. This finding led to a considerable amount of debate, and a number of complex processing models have been proposed as an explanation. What these models have in common is that they postulate two or more separate processing streams, in order to reconcile the Semantic Illusion and other semantically induced P600 effects with the traditional interpretations of the N400 and the P600. Recently, however, these multi-stream models have been called into question, and a simpler single-stream model has been proposed. According to this alternative model, the N400 component reflects the retrieval of word meaning from semantic memory, and the P600 component indexes the integration of this meaning into the unfolding utterance interpretation. In the present paper, we provide support for this "Retrieval-Integration (RI)" account by instantiating it as a neurocomputational model. This neurocomputational model is the first to successfully simulate the N400 and P600 amplitude in language comprehension, and simulations with this model provide a proof of concept of the single-stream RI account of semantically induced patterns of N400 and P600 modulations.

Keywords: Computational modeling; Event-related potentials; Language comprehension; N400; Neural networks; P600; Retrieval-Integration account.

Figure 1

Results (Pz electrode) of the ERP experiment by Hoeks et al. (2004). Positive is plotted upward. Note that this single electrode only serves as an illustration; our present simulation results are compared to the (statistically evaluated) effects found on the whole array of electrodes used in the original study.

Figure 2

Schematic illustration of the neurocomputational model instantiating the Retrieval–Integration account of the N400 and the P600. Each rectangle represents a vector of artificial (logistic dot‐product) neurons, and each solid arrow represents a matrix of connection weights that connect each neuron in a projecting layer to each neuron in a receiving layer. The network receives its input at the input layer and produces its output at the integration_output layer. The rectangle with a dashed border is a context layer (cf. Elman, 1990), and the dashed arrow represents a copy projection; prior to feedforward propagation of activation from the input to the integration_output layer, the integration_context layer receives a copy of the integration layer. At time step t = 0, the activation value of each unit in the integration_context layer is set to 0.5. All layers except the input and integration_context layer also receive input from a bias unit (not shown), the activation value of which is always 1.

Figure 3

Illustration of the word‐by‐word processing of an example sentence (from simulation 1) for each condition of the Hoeks et al. (2004) experiment (see text). The bar plots show the cosine similarity of the word meaning representation in each thematic‐role slot (in the integration_output layer) relative to either the representation of each of the nouns (“kok”/“cook” and “maaltijd”/“meal”) for the agent and patient slots, or to that of each of the verbs (“bereid”/“prepared” and “gezongen”/“sung”) for the action slot.

Figure 4

N400 results of the simulations in comparison to the results of the original experiment by Hoeks et al. (2004). Panel (A) shows the N400 amplitudes as measured in the original experiment (at the Pz electrode), transformed to a zero‐to‐one scale (see text). Panel (B) shows the N400 estimates measured in simulation 1, and panel (C) those measured in simulation 2. Error bars show standard errors.

Figure 5

P600 results of the simulations in comparison to the results of the original experiment by Hoeks et al. (2004). Panel (A) shows the P600 amplitudes as measured in the original experiment (at the Pz electrode), transformed to a zero‐to‐one scale (see text). Panel (B) shows these P600 amplitudes corrected for overlap with the N400 component (also at Pz and on a zero‐to‐one scale). Panel (C) shows the P600 estimates measured in simulation 1, and panel (D) those measured in simulation 2. Error bars show standard errors.

Figure 6

Schematic illustration of the functional‐anatomic mapping of the Retrieval–Integration account onto the core language network (top), and its relation to the neurocomputational model (bottom). Note that the schematic of the model uses a shorthand notation for the contextual input to the retrieval and integration layers (by omitting the integration_context layer). An incoming word reaches the lpMTG via either the auditory cortex (ac) or visual cortex (vc) (corresponding to the input layer in the model). The lpMTG then retrieves the conceptual knowledge associated with this word from the association cortices (retrieval→ retrieval_output), thereby generating the N400. Next, this retrieved meaning is sent to the lIFG (retrieval_output→ integration), where it is integrated with its prior context (integration → integration) into an updated utterance representation (integration → integration_output). This integration process is reflected in the P600 component. The updated utterance representation in the lIFG subsequently provides a context for the retrieval of the conceptual knowledge associated with the next word (integration → retrieval).