Invariants for neural automata

Abstract

Computational modeling of neurodynamical systems often deploys neural networks and symbolic dynamics. One particular way for combining these approaches within a framework called vector symbolic architectures leads to neural automata. Specifically, neural automata result from the assignment of symbols and symbol strings to numbers, known as Gödel encoding. Under this assignment, symbolic computation becomes represented by trajectories of state vectors in a real phase space, that allows for statistical correlation analyses with real-world measurements and experimental data. However, these assignments are usually completely arbitrary. Hence, it makes sense to address the problem which aspects of the dynamics observed under a Gödel representation is intrinsic to the dynamics and which are not. In this study, we develop a formally rigorous mathematical framework for the investigation of symmetries and invariants of neural automata under different encodings. As a central concept we define patterns of equality for such systems. We consider different macroscopic observables, such as the mean activation level of the neural network, and ask for their invariance properties. Our main result shows that only step functions that are defined over those patterns of equality are invariant under symbolic recodings, while the mean activation, e.g., is not. Our work could be of substantial importance for related regression studies of real-world measurements with neurosymbolic processors for avoiding confounding results that are dependant on a particular encoding and not intrinsic to the dynamics.

Keywords: Computational cognitive neurodynamics; Invariants; Language processing; Neural automata; Observables; Symbolic dynamics.

Fig. 1

The vocabulary ${\mathbf{A}}^{\ast }$ as a rooted tree

Fig. 2

The regular rooted tree T over the alphabet $\{0,1,\cdots ,m-1\}$

Fig. 3

Cylinder set corresponding to w seen on the tree

Fig. 4

Invariant partition of the cylinder sets according to their patterns of equality

Fig. 5

Invariant partition of the interval [0, 1] after Gödelization

Fig. 6

Each small square corresponds to a square on the partition given by the dotted sequences of length (2, 3). The squares colored by the same color are those having the same pattern of equality, and thus, are those which can be mapped to each other under different Gödel encodings of the alphabet

Fig. 7

The macroscopic observable f, given by the step function (24) is invariant under Gödel recoding. The figure shows the result of ‘measuring’ f to a neural automaton encoded by $\gamma$ on top and to the same machine encoded by $\delta$ below

Fig. 8

Amari’s mean-field observable Eq. (9) of the neural automaton under two different Gödel encodings $\gamma$ and $\delta$