The Neurally Controlled Animat: Biological Brains Acting with Simulated Bodies

Abstract

The brain is perhaps the most advanced and robust computation system known. We are creating a method to study how information is processed and encoded in living cultured neuronal networks by interfacing them to a computer-generated animal, the Neurally-Controlled Animat, within a virtual world. Cortical neurons from rats are dissociated and cultured on a surface containing a grid of electrodes (multi-electrode arrays, or MEAs) capable of both recording and stimulating neural activity. Distributed patterns of neural activity are used to control the behavior of the Animat in a simulated environment. The computer acts as its sensory system providing electrical feedback to the network about the Animat's movement within its environment. Changes in the Animat's behavior due to interaction with its surroundings are studied in concert with the biological processes (e.g., neural plasticity) that produced those changes, to understand how information is processed and encoded within a living neural network. Thus, we have created a hybrid real-time processing engine and control system that consists of living, electronic, and simulated components. Eventually this approach may be applied to controlling robotic devices, or lead to better real-time silicon-based information processing and control algorithms that are fault tolerant and can repair themselves.

Figure 1

Scheme for the Neurally Controlled Animat. A network of hundreds or thousands of dissociated mammalian cortical cells (neurons and glia) are cultured on a transparent multi-electrode array. Their activity is recorded extracellularly to control the behavior of an artificial animal (the Animat) within a simulated environment. Sensory input to the Animat is translated into patterns of electrical stimuli sent back into the network.

Figure 2

(A) A culture of live neurons after 3 days in culture on a 8 × 8 grid of 60- 10 μm-electrodes separated by 200 μm. (B) Expanded view of the lower left hand corner of the array showing the neurons and axons that form the living neural network. The large circular pads are the recording/stimulating electrodes that detect the extracellular neural activity of neurons located on or near each electrode. Insulated traces (dark lines in the figure) interface each electrode to amplifiers and A/D converters. MEAs and recording electronics are from MultiChannel Systems. (C) Raw signal from one electrode showing action potentials and background noise (signal to noise of 20 or more, typically). Spikes (action potentials) are detected in real-time as voltage peaks above a threshold of 5 × standard deviation of the signal amplitude, calculated independently for each channel.

Figure 3

Movement and results of the session in which the Animat was online. Top left: Trajectory of movement of the Animat in a simulated room. Top right: The frequency that different patterns would recur. Bottom left: Examples of the four most common patterns. Bottom right: Number of novel patterns that the clustering algorithm detected as the session progressed (total 51). Bottom left: Examples of the four most common patterns.