Tensors and compositionality in neural systems

Abstract

Neither neurobiological nor process models of meaning composition specify the operator through which constituent parts are bound together into compositional structures. In this paper, we argue that a neurophysiological computation system cannot achieve the compositionality exhibited in human thought and language if it were to rely on a multiplicative operator to perform binding, as the tensor product (TP)-based systems that have been widely adopted in cognitive science, neuroscience and artificial intelligence do. We show via simulation and two behavioural experiments that TPs violate variable-value independence, but human behaviour does not. Specifically, TPs fail to capture that in the statements fuzzy cactus and fuzzy penguin, both cactus and penguin are predicated by fuzzy(x) and belong to the set of fuzzy things, rendering these arguments similar to each other. Consistent with that thesis, people judged arguments that shared the same role to be similar, even when those arguments themselves (e.g., cacti and penguins) were judged to be dissimilar when in isolation. By contrast, the similarity of the TPs representing fuzzy(cactus) and fuzzy(penguin) was determined by the similarity of the arguments, which in this case approaches zero. Based on these results, we argue that neural systems that use TPs for binding cannot approximate how the human mind and brain represent compositional information during processing. We describe a contrasting binding mechanism that any physiological or artificial neural system could use to maintain independence between a role and its argument, a prerequisite for compositionality and, thus, for instantiating the expressive power of human thought and language in a neural system. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

Keywords: binding; compositionality; concepts; language; predicates; tensor products.

Figure 1.

An illustration of two representational coding schemes for the predicates fuzzy (penguin), fuzzy (cactus), prickly (penguin) and prickly (cactus). In a role–filler binding calculus, these propositions can be represented as one-place predicates. The top panel is a cartoon illustration of how a compositional system that uses dynamic binding would bind the predicates and arguments. The bottom panel is a cartoon illustration, inspired by Dolan & Smolensky [23], of how a tensor-based system would perform binding.

Figure 2.

A snapshot of neural space–time of ADB of fuzzy penguin prickly cactus, fuzzy cactus and prickly penguin using the same constituent representations and composing them through dynamic binding with a features layer and a separate conjunctive coding layer.