Harnessing chaos in recurrent neural networks

Abstract

In this issue of Neuron, Sussillo and Abbott describe a new learning rule that helps harness the computational power of recurrent neural networks.

Figure 1

A. Complex self-maintaining activity patterns are observed in response to a brief stimulus (gray) in a recurrent network (ellipse on left, with blue circles representing neurons and arrows synapses) in which the weights are randomly assigned strong values (g=1.5). Each line represents the activity of a single unit of a large recurrent network. The dashed lines represent the same simulation in which the activity of a single unit was altered at t=20ms. The divergence indicates a high-sensitivity to noise, suggestive of chaotic behavior. B. FORCE learning rule applied to a network with g=1.5 and trained to generate a 10 Hz sinusoid, at the onset of a brief input (gray, there was also an offset signal at t = 1 sec). Dashed lines represent the same simulation when the activity of a single units was altered at t = −750 ms. This network includes an external feedback unit which receives inputs (red) from the recurrent network. Only the W_Out (red) were modified during training.