Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Feb 20;24(3):102222.
doi: 10.1016/j.isci.2021.102222. eCollection 2021 Mar 19.

EqSpike: spike-driven equilibrium propagation for neuromorphic implementations

Erwann Martin  1 , Maxence Ernoult  2   3 , Jérémie Laydevant  2 , Shuai Li  2 , Damien Querlioz  3 , Teodora Petrisor  1 , Julie Grollier  2
Affiliations

EqSpike: spike-driven equilibrium propagation for neuromorphic implementations

Erwann Martin et al. iScience. .

Abstract

Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge. Equilibrium propagation is a promising alternative to backpropagation as it only involves local computations, but hardware-oriented studies have so far focused on rate-based networks. In this work, we develop a spiking neural network algorithm called EqSpike, compatible with neuromorphic systems, which learns by equilibrium propagation. Through simulations, we obtain a test recognition accuracy of 97.6% on the MNIST handwritten digits dataset (Mixed National Institute of Standards and Technology), similar to rate-based equilibrium propagation, and comparing favorably to alternative learning techniques for spiking neural networks. We show that EqSpike implemented in silicon neuromorphic technology could reduce the energy consumption of inference and training, respectively, by three orders and two orders of magnitude compared to graphics processing units. Finally, we also show that during learning, EqSpike weight updates exhibit a form of spike-timing-dependent plasticity, highlighting a possible connection with biology.

Keywords: Algorithms; Artificial Intelligence; Computer Science.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

None
Graphical abstract
Figure 1
Figure 1
EqSpike: spike-driven equilibrium propagation (A) Schematic of the free phase in equilibrium propagation. (B) Schematic of the nudging phase in equilibrium propagation. (C) Illustration of the weight update implementation in EqSpike. (D) Spiking rate of the neuron as a function of the amplitude of the input signal. (E) Schematic of the rate acceleration computation.
Figure 2
Figure 2
Recognition accuracy during training (orange) and test (blue) as a function of the number of epochs for MNIST, averaged over six runs. The error bars, in light color, correspond to the standard deviation over the six runs.
Figure 3
Figure 3
Inference time: recognition accuracy on MNIST on the test data set as a function of time multiplied by the maximum neuron frequency fmax Orange line: recognition accuracy computed from the output neuron showing the highest rate. Blue line: recognition accuracy computed from the output neuron spiking first. Red, vertical dotted line: average time of first output spike. The width of the lines in light colors correspond to the standard deviation over the different runs.
Figure 4
Figure 4
Training performance (A) Number of presented images in the nudging phase per epoch versus epoch number. (B) Number of spikes/neuron/image occurring during the two phases (nudge + free), as a function of the epoch. (C) SynOps: number of spikes during both phases (nudge + free) as a function of the recognition accuracy.
Figure 5
Figure 5
STDP (A) Illustration of the STDP learning rule, reproduced with data from (Bi and Poo, 2001). (B) Illustration of the link between Eq-Prop and STDP learning rules; illustration reproduced from (Bengio et al., 2017). (C) STDP-like curve during EqSpike learning.
See this image and copyright information in PMC

References

    1. Bellec G., Scherr F., Subramoney A., Hajek E., Salaj D., Legenstein R., Maass W. A solution to the learning dilemma for recurrent networks of spiking neurons. Nat. Commun. 2020;11:3625. - PMC - PubMed
    1. Bengio Y., Mesnard T., Fischer A., Zhang S., Wu Y. STDP-compatible approximation of backpropagation in an energy-based model. Neural Comput. 2017;29:555–577. - PubMed
    1. Bi G., Poo M. Synaptic modification by correlated activity: hebb’s postulate revisited. Annu. Rev. Neurosci. 2001;24:139–166. - PubMed
    1. Bichler O., Querlioz D., Thorpe S., J., Bourgoin J.-P., Gamrat C. Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity. Neural Networks. 2012;32:339–348. - PubMed
    1. Davies M., Srinivasa N., Lin T.-H., Chinya G., Cao Y., Choday S., H., Dimou G., Joshi P., Imam N., Jain S. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro. 2018;38:82–99.

LinkOut - more resources