Concepts, control, and context: A connectionist account of normal and disordered semantic cognition

Abstract

Semantic cognition requires conceptual representations shaped by verbal and nonverbal experience and executive control processes that regulate activation of knowledge to meet current situational demands. A complete model must also account for the representation of concrete and abstract words, of taxonomic and associative relationships, and for the role of context in shaping meaning. We present the first major attempt to assimilate all of these elements within a unified, implemented computational framework. Our model combines a hub-and-spoke architecture with a buffer that allows its state to be influenced by prior context. This hybrid structure integrates the view, from cognitive neuroscience, that concepts are grounded in sensory-motor representation with the view, from computational linguistics, that knowledge is shaped by patterns of lexical co-occurrence. The model successfully codes knowledge for abstract and concrete words, associative and taxonomic relationships, and the multiple meanings of homonyms, within a single representational space. Knowledge of abstract words is acquired through (a) their patterns of co-occurrence with other words and (b) acquired embodiment, whereby they become indirectly associated with the perceptual features of co-occurring concrete words. The model accounts for executive influences on semantics by including a controlled retrieval mechanism that provides top-down input to amplify weak semantic relationships. The representational and control elements of the model can be damaged independently, and the consequences of such damage closely replicate effects seen in neuropsychological patients with loss of semantic representation versus control processes. Thus, the model provides a wide-ranging and neurally plausible account of normal and impaired semantic cognition. (PsycINFO Database Record

Figure 1

Architecture of the representational model. Black layers comprise visible units that receive inputs and/or targets from the environment. Gray layers represent hidden units. Solid arrows indicate full, trainable connectivity between layers. The dashed arrow represents a copy function whereby, following processing of a stimulus, the activation pattern over the hub layer is replicated on the context layer where it remains to act as the context for the next stimulus.

Figure 2

An example episode. The 10 inputs for the episode are shown from left to right, along with the targets provided at each point. For example, at the first point in this sequence, the verbal input unit for car is activated and the model is trained to turn on the S-M units associated with cars and the prediction unit for journey (as this is the next item in the sequence). <ITEM > represents the S-M properties of a concrete item.

Figure 3

The model’s vocabulary.

Figure 4

Example topic distributions. Concepts with S-M features are shown in italics. The PETROL STATION topic was used to generate the episode shown in Figure 2.

Figure 5

Context-sensitive representation of the word pump. The model was presented with pump immediately following either truck, shoe, or deposit. Results are averaged over 50 such presentations. Left: Activation of prediction units, indicating that the model’s expectations change when the word appears in these different contexts. Right: Results of multidimensional scaling analyses performed on the hub representations of words presented in each context. In these plots, the proximity of two words indicates the similarity of their representations over the hub units (where similarity is measured by the correlation between their activation vectors). The model’s internal representation of pump shifts as a function of context.

Figure 6

Hub representations of concrete and abstract concepts. Concrete concepts are color-coded by category. Abstract concepts are shown in greyscale, where shading indicates pairs of semantically related words.

Figure 7

S-M unit activations for a selection of concrete and abstract words. (A) Activations of S-M units shared by the members of each category, in response to a selection of words. Each word was presented to the network 50 times (with a different random pattern of activity on the context units) and the results averaged to generate this figure. (B) Activation of S-M units in response to the same abstract word in two different contexts.

Figure 8

Example trials from the homonym comprehension task.

Figure 9

Target data and model performance for Simulation 1.

Figure 10

Activation of response options in the model with no control processes. The bars in the bottom right corner of each plot show the standard deviation of the Gaussian function used to add noise to each activation.

Figure 11

The controlled retrieval process. (A) The model is asked to decide which of four alternatives is most semantically related to bank. (B) Input to the model during settling. The model receives sustained input of the probe and a weighted combination of the possible responses. As the prediction for river strengthens, it comes to dominate the input. (C). Activation of prediction units during settling. The controlled retrieval process boosts the activation of river, relative to the level it would receive from processing of the probe alone (dashed line). (D) Graphical representation of settling. Elements of the controlled retrieval mechanism are shown in red.

Figure 12

Model’s trajectory through semantic space during the bank trial. This plot illustrates the effect of controlled retrieval on the model’s internal representations. We first presented each word to the model in turn, allowed it to settle and recorded activity of the hub layer. Multidimensional scaling was used to plot the relationships between these states in a two-dimensional space (words used in the current trial are highlighted). We then recorded the activity of the hub units as the model completed the dominant and subordinate versions of the bank trial with and without controlled retrieval (CR). The lines plot the trajectory taken by the model through the semantic space as it settled. Without controlled retrieval, settling is determined solely by the identity of the probe, resulting in similar paths on dominant and subordinate trials, both of which end near the canonical representation of bank. Under controlled retrieval, settling is constrained by both probe and target. As a consequence, the model is deflected into areas of the semantic space somewhere between bank and either cashier or tree.

Figure 13

Model performance in Simulation 1 under alternative forms of damage.

Figure 14

Performance of the intact model in Simulation 1 with an alternative form of controlled retrieval.

Figure 15

Target data and model performance for Simulation 2. (A) Accuracy levels for human data and in the model (healthy control data taken from Hoffman et al., 2013b; patient data from Hoffman et al., 2011b). (B) Beta values from linear regression models that used psycholinguistic properties to predict human and model performance on individual trials.

Figure 16

Target data and model performance for Simulation 3.