Modeling the Synchronization of Multimodal Perceptions as a Basis for the Emergence of Deterministic Behaviors

Abstract

Living organisms have either innate or acquired mechanisms for reacting to percepts with an appropriate behavior e.g., by escaping from the source of a perception detected as threat, or conversely by approaching a target perceived as potential food. In the case of artifacts, such capabilities must be built in through either wired connections or software. The problem addressed here is to define a neural basis for such behaviors to be possibly learned by bio-inspired artifacts. Toward this end, a thought experiment involving an autonomous vehicle is first simulated as a random search. The stochastic decision tree that drives this behavior is then transformed into a plastic neuronal circuit. This leads the vehicle to adopt a deterministic behavior by learning and applying a causality rule just as a conscious human driver would do. From there, a principle of using synchronized multimodal perceptions in association with the Hebb principle of wiring together neuronal cells is induced. This overall framework is implemented as a virtual machine i.e., a concept widely used in software engineering. It is argued that such an interface situated at a meso-scale level between abstracted micro-circuits representing synaptic plasticity, on one hand, and that of the emergence of behaviors, on the other, allows for a strict delineation of successive levels of complexity. More specifically, isolating levels allows for simulating yet unknown processes of cognition independently of their underlying neurological grounding.

Keywords: behavioral learning; developmental cognition; neural circuit; synchronized perceptions; virtual machine.

Figure 1

High level definition of a virtual machine run.

Figure 2

Circuit fragment implementing a synaptic transmission. A signal is transferred between P and Q, and ->=>- depicts a synapse.

Figure 3

Thread patterns for a synaptic transmission. Thread P will fire in reaction to the capture of an external stimulus, with the send procedure corresponding to sending a signal, or spike train, carried by a pre-synaptic neuron's axon. In the thread Q, the receive procedure represents the reception of this signal by a post-synaptic neuron.

Figure 4

Communication protocol for an asynchronous communication. The send/receive protocol corresponds to an asynchronous communication subject to a threshold. It involves a predefined weight between the sender P and the receiver Q. This weight can be incremented/decremented by an ltp/ltd thread (see below). After firing thread Q and sending it a signal, thread P goes on executing its next instruction. On the other side, thread Q waits for the reception of a signal from thread P and proceeds only if the weight between P and Q stands above a given threshold.

Figure 5

A circuit implementing classical conditioning. The symbol $\text{/|}$ represents the modulation of a synaptic transmission, the sign * used in the upper branch indicates the conjunction of converging signals, and the sign + either the splitting of a diverging signal, as used in the lower branch, or a choice between converging signals, as used in the right branch instantiating the thread motor(X), where X is a variable parameter to be instantiated into either cs or us. The thread $\text{ltp}$ (standing for long term potentiation) acts as a facilitatory interneuron reinforcing the pathway between sense(cs) and motor(cs).

Figure 6

Micro-circuit and communication protocol for ltp. In order to detect the coincidence of P and Q, thread P fires an ltp thread that in turn calls on a join procedure to wait for a signal from thread Q. In parallel, thread Q calls on procedure merge to post a signal for ltp and then executes a send® command to establish a link with thread R. After its synchronization with thread Q, thread ltp increments the weight between Q and R (NB This protocol will be at the heart of the developments to come and will allow for the learning of causality rules).

Figure 7

Experiment configuration. The vehicle is at home, $\text{smoke}$ at $\text{right}$, and a $\text{fire}$ at $\text{right(3)}$.

Figure 8

Vehicle two possible moves: (A) the vehicle moved $\text{backward}$ to the $\text{left}$ and missed the $\text{fire}$, (B) the vehicle moved $\text{forward}$ to the $\text{right}$ and cleared the $\text{fire}$.

Figure 9

Circuit implementing a random detector. In this circuit, (1) detecting the smoke of a fire at location F(X) is implemented by the thread sense(F(X)), (2) the random choice between two gears A or B follows from the execution of the virtual machine instruction choice([A,B])that produces an internal fetch stimulus, (3) detecting the fire proceeds through the thread $\text{detect(F(X))}$, with the execution of the virtual machine instruction $\text{check(on(F(X)))}$ producing an $\text{excite}$ internal stimulus if the vehicle did hit the fire (in which case he will $\text{clear}$ the fire) or an $\text{inhibit}$ internal stimulus (in which case he will $\text{resume}$ detecting) (4) moving is achieved through the $\text{move(A)}$ or $\text{move(B)}$ threads; depending on whether or not the current vehicle position $\text{F(X)}$ is defined, the execution of the virtual machine instruction check(at(F(X))) similarly produces an internal $\text{excite}$ (in which case he will $\text{resume}$ moving) or an $\text{inhibit}$ stimulus indicating that he has reached the track limit and must $\text{stop}$.

Figure 10

Execution trace of the vehicle randomly moving in the wrong direction. Following an incorrect fetch(backward) random choice, the $\text{detect}$ and $\text{move}$ concurrently produced a series of $\text{inhibit}$ and $\text{excite}$ stimuli until a single $\text{inhibit}$ produced by the $\text{move}$ thread at time 9 (signaling that the vehicle had reached the track limit) forced him to stop. At the same time, the $\text{detect}$ threads went on producing $\text{inhibit}$ stimuli (signaling that the fire had not been detected, and the detector not yet turned off).

Figure 11

Execution trace of the vehicle randomly moving into the correct direction. Following a correct fetch(forward) random choice, two simultaneous $\text{excite}$ internal stimuli indicate that two synchronized perceptions occurred, namely the detection of a fire at the vehicle's position. Subsequently the fire was cleared.

Figure 12

Neural circuit implementing a simple form operant conditioning. At the beginning, the pathway from sense to learn is open, while the pathways to $\text{accept}$ and $\text{reject}$ are closed. The learn thread discriminates between positive and negative external stimuli, and thus functions as a fork directing synaptic plasticity. Synaptic plasticity is expressed through $\text{ltp/ltd}$ threads and eventually leads to open the path to either $\text{accept}$ or $\text{reject}$, and close the path to learn.

Figure 13

Circuit implementing a deterministic behavior. Let Y represent A or B. The $\text{ltp/ltd}$ threads are activated via $\text{recall(Y)}$ threads by synchronization threads $\text{synchro(detect(F(X)),move(Y))}$, which themselves fire whenever the tests $\text{check(on(F(X)))}$in $\text{detect(F(X))}$and $\text{check(at(F(X)))}$in $\text{move(Y)}$ provide a simultaneous $\text{excite}$ stimulus.

Figure 14

Execution trace of learning a single move. Facing an open path to learn, a random choice learn(right):fetch(forward) successfully led to find the fire at position right(3), activating a $\text{synchro}$ thread that initiated $\text{ltp/ltd}$ threads (see Figure 5 for the communication protocol of $\text{ltp/ltd}$), thus inducing synaptic weight changes into instantiated links and leading in turn to both open the path to $\text{(sense(right(3)),recall(forward))}$ and close the path to $\text{(sense(right(3)),learn(right))}$.

Figure 15

Execution trace of a repeated move. Facing a closed path to learn, the vehicle did proceed without fetch stimulus (but with a repeated $\text{inhibit}$ at time 1), and went through the open path of $\text{recall}$ to detect a fire at the same position as before.

Figure 16

Execution trace of failing to detect a new fire: Facing an open path to learn, the vehicle made a random choice learn(right):fetch(backward) i.e., an incorrect choice that did not lead him to detect the fire.

Figure 17

Execution trace of learning a non instantiated rule. This run is similar to the one presented in Figure 14, with the only difference being the weight changes applied to non instantiated links i.e., $\text{(sense(right(\_)),recall(forward))}$ and $\text{(sense(right(\_)),learn(right))}$.

Figure 18

Execution trace of applying a rule. This run is similar to the one of Figure 15, but with a fire at different location in the same direction.