Bi-sigmoid spike-timing dependent plasticity learning rule for magnetic tunnel junction-based SNN

Abstract

In this study, we explore spintronic synapses composed of several Magnetic Tunnel Junctions (MTJs), leveraging their attractive characteristics such as endurance, nonvolatility, stochasticity, and energy efficiency for hardware implementation of unsupervised neuromorphic systems. Spiking Neural Networks (SNNs) running on dedicated hardware are suitable for edge computing and IoT devices where continuous online learning and energy efficiency are important characteristics. We focus in this work on synaptic plasticity by conducting comprehensive electrical simulations to optimize the MTJ-based synapse design and find the accurate neuronal pulses that are responsible for the Spike Timing Dependent Plasticity (STDP) behavior. Most proposals in the literature are based on hardware-independent algorithms that require the network to store the spiking history to be able to update the weights accordingly. In this work, we developed a new learning rule, the Bi-Sigmoid STDP (B2STDP), which originates from the physical properties of MTJs. This rule enables immediate synaptic plasticity based on neuronal activity, leveraging in-memory computing. Finally, the integration of this learning approach within an SNN framework leads to a 91.71% accuracy in unsupervised image classification, demonstrating the potential of MTJ-based synapses for effective online learning in hardware-implemented SNNs.

Keywords: MTJ; SNN; STDP; neuromorphic; online learning; spintronics; unsupervised.

Figure 1

(A) A schematic of an MTJ device composed of two ferromagnetic layers which are separated by an insulating layer. on the right is shown the switching mechanism from parallel to anti-parallel (P2AP) and vice versa (AP2P) depending on the current polarity. (B) An SNN network composed of the synaptic crossbar, input, and output neurons emitting their specific pulses. Each synapse is a set of multiple MTJs connected in parallel to enable a multi-level conductance synapse.

Figure 2

Statistical simulations showing the required voltage (amplitude, duration) to switch the MTJ state for both potentiation (AP2P) and depression(P2AP). For each voltage amplitude, we run 100 simulations to get the distribution of the required pulse width durations.

Figure 3

Biological STDP vs. Bi-sigmoid STDP, the weight update in the function of the time difference between pre- and post-synaptic pulses.

Figure 4

Temporal relationship between pre- and post-synaptic pulses (V_pre and V_post), followed by the corresponding synaptic update. (A) 4MTJ synapse subject to the voltage drop V_pre − V_post. (B) and (C) depict V_pre and a sequence of V_post respectively. (D) shows different scenarios of the resulting voltage drop across the synapse when V_post arrives at different delays (t1 to t7) relative to V_pre. (E) provides a normalized view of the synaptic conductance update, highlighting periods of potentiation and depression corresponding to the timing sequences t1 to t7.

Figure 5

Probability of switching of an MTJ at various pulse widths, when the voltage is varied within the Neel-Brown interval. (A): Depression, (B): Potentiation.

Figure 6

Synaptic design exploration based on electrical simulations. (A, B) Show the influence of two different pre-synaptic pulse widths on potentiation and depression curves respectively, for each V_pre, V_post arrives at different delays, and the subsequent synaptic update is observed. (C, D) Present the effects of varying the number of MTJs per synapse in its conductance, for potentiation and depression respectively.

Figure 7

(A) Normalized synaptic weight updates relative to the timing differences between pre- and post-synaptic spikes. Spice simulation data fitted with the Bi-sigmoid function. (B) Label distribution map for each excitatory neuron after training, indicating learned representations. (C) Incremental improvement in classification performance as a function of the number of training samples. (D) Visualization of synaptic weights between the input layer and 100 excitatory neurons, reshaped into 28x28 matrices, each representing a learned digit from the MNIST dataset.