Spatial Memory in a Spiking Neural Network with Robot Embodiment

Abstract

Cognitive maps and spatial memory are fundamental paradigms of brain functioning. Here, we present a spiking neural network (SNN) capable of generating an internal representation of the external environment and implementing spatial memory. The SNN initially has a non-specific architecture, which is then shaped by Hebbian-type synaptic plasticity. The network receives stimuli at specific loci, while the memory retrieval operates as a functional SNN response in the form of population bursts. The SNN function is explored through its embodiment in a robot moving in an arena with safe and dangerous zones. We propose a measure of the global network memory using the synaptic vector field approach to validate results and calculate information characteristics, including learning curves. We show that after training, the SNN can effectively control the robot's cognitive behavior, allowing it to avoid dangerous regions in the arena. However, the learning is not perfect. The robot eventually visits dangerous areas. Such behavior, also observed in animals, enables relearning in time-evolving environments. If a dangerous zone moves into another place, the SNN remaps positive and negative areas, allowing escaping the catastrophic interference phenomenon known for some AI architectures. Thus, the robot adapts to changing world.

Keywords: STDP; cognitive maps; learning; neurorobotics; spiking neural networks; vector field of functional connections; vector field of synaptic connections.

Figure 1

Examples of vector fields defined by synaptic connections in four small neural assemblies. Black arrows show the direction and magnitude of the synaptic vector field in a grid containing 7 × 23 cells (black points correspond to zero vectors). The thickness of the red links connecting neurons is proportional to the weight of interneuron couplings.

Figure 2

Emergence of traveling patches of neural activity and their relation with the vector field of functional connections (black arrow). (A) Spontaneous firing before stimulation (red and blue circles mark excited excitatory and inhibitory neurons, respectively; the time instants t_b of the network burst refers to the interval past from the stimulus onset). (B) Network activity during periodic local stimulation. (C) Spontaneous activity after long-term stimulation.

Figure 3

Comparison of the vector fields of anatomical (black arrows) and functional (red arrow) connections during neuronal activity before (A), during (B), and after (C) a period of long-term local stimulation. The intensity of the magenta color of the cells is proportional to the field difference.

Figure 4

Characteristics of global network memory. (A) Top: The vector field of the synaptic weights (thin black arrows) of a segment of the neural network before external stimulation (only the first quadrant of the network is shown). The thick black arrow, g_c, shows the global connectivity vector of the selected segment. The magenta-framed inset shows incoming couplings for a representative neuron (line thickness corresponds to the coupling weight). Bottom: A typical epoch of the raster plot during the stimulation (each point displays a spike of the corresponding neuron; neurons have been sorted by their distance to the stimulation site). (B) Same as in (A) but immediately after external stimulation. The thick red arrow, g_st, is the vector of global memory after stimulation. (C) Same as in (A) but after a period of forgetting. The cosine of the angle between the connectivity vector at the current time g_c (black) and the vector after stimulation g_st (dashed, red) is the measure of global network memory.

Figure 5

Correlation of the learning curve with the global network memory measure. (A) Averaged relative synchronization time (learning curve) and memory measure M as functions of the stimulus number. The time interval between successive stimulations is ${10}^4-3\times {10}^4$ s, the noise intensity D = 4.8 (n = 10). The relative synchronization time is the time required to synchronize the network, normalized to the synchronization time at the first stimulation. M was evaluated just before the stimulation. (B) Same as in (A) but for the increased time interval between stimulations ($5\times {10}^4-{10}^5$ s) and the intensity of neural noise (D = 5.1–5.5; n = 10). (C) Synchronization time versus the global memory measure (n = 200).

Figure 6

Embodiment of the SNN in a moving robot. (A) The robot is controlled by the SNN and moves in a square arena divided into safe and dangerous (marked with a grid) zones. (B) The robot location determines the stimulation area in the network space (red circle). The stimulation frequency is 1 and 10 Hz for the safe and dangerous zones, respectively. (C) The vector field of functional connections (red arrows) in the SNN determines the robot’s direction and speed (red circle delimits the area controlling the robot movement).

Figure 7

Adapting the robot behavior to changing environment. (A) Top: Traces of the robot movements in the arena without dangerous zones (all four quadrants are safe). The robot randomly explored the environment. Bottom: Moving average of the time spent by the robot in quadrants I (blue curve) and III (red curve). (B) Same as in (A) but during movement in the arena with the danger zone in quadrant III. The robot spent little time in the dangerous quadrant (red curve), while the time spent in quadrant I rose (blue curve). (C) Same as in (B) but during relearning. The dangerous zone had moved from quadrant III to I. The robot learned the new dangerous area and recovered the previous one that was now safe. The times spent by the robot in quadrants I and III were inverted (the blue curve decayed, while the red one increased).