Single neuromorphic memristor closely emulates multiple synaptic mechanisms for energy efficient neural networks

Abstract

Biological neural networks do not only include long-term memory and weight multiplication capabilities, as commonly assumed in artificial neural networks, but also more complex functions such as short-term memory, short-term plasticity, and meta-plasticity - all collocated within each synapse. Here, we demonstrate memristive nano-devices based on SrTiO₃ that inherently emulate all these synaptic functions. These memristors operate in a non-filamentary, low conductance regime, which enables stable and energy efficient operation. They can act as multi-functional hardware synapses in a class of bio-inspired deep neural networks (DNN) that make use of both long- and short-term synaptic dynamics and are capable of meta-learning or learning-to-learn. The resulting bio-inspired DNN is then trained to play the video game Atari Pong, a complex reinforcement learning task in a dynamic environment. Our analysis shows that the energy consumption of the DNN with multi-functional memristive synapses decreases by about two orders of magnitude as compared to a pure GPU implementation. Based on this finding, we infer that memristive devices with a better emulation of the synaptic functionalities do not only broaden the applicability of neuromorphic computing, but could also improve the performance and energy costs of certain artificial intelligence applications.

Fig. 1. Biologically inspired synaptic functions and their memristor implementation.

a Organization of the mammalian brain with several biological neurons connected through synapses. When a postsynaptic spike (light blue) coincides with a presynaptic spike (light green) the corresponding synaptic coupling is strengthened (Hebbian plasticity) for a limited amount of time (short-term plasticity). This bio-physical process is illustrated in the circular insets: (I) an influx of ions (e.g., Ca²⁺) through the postsynaptic voltage gated ion channels leads to (II) an increased number of synaptic receptors, which increases the synaptic weight. (III) The weight subsequently decays back to its original value due to the receptors gradually detaching from the membrane. b Table comparing the synaptic functions of artificial synapses in standard ANNs (Artificial column) and biological synapses (Biological column). The plot on the right shows the weight of a biological synapse as a function of time. The short-term weight (F) is updated (ΔF) when the pre- and post-synaptic spikes coincide. Additionally, the decay time of F can be controlled, which corresponds to meta-plasticity. c Bio-inspired Short-Term Plasticity Neuron (STPN) model combining a conventional neuron model with short-term Hebbian (ST-Hebb) synapses. d Hardware implementation of a neuromorphic ST-Hebb synapse with a Cr/Pt-SrTiO₃-Ti memristor. The device measurement on the right mirrors the biological functions of ST-Hebb synapses combining memory and computation as well as long- (W) and short-term (F) dynamics.

Fig. 2. DC and dynamical behavior of multi-functional memristive synapses.

a Conductance vs. voltage characteristic (30 cycles) of the fabricated Cr/Pt-STO-Ti memristors. The black arrows indicate the counter-clockwise switching direction. b Sketch of the device stack and of the underlying switching mechanism. The two insets zoom into the Pt-STO interface at different applied voltages, showing the dynamics of interfacial oxygen ions (O) and oxygen vacancies (${\mathrm{V}}_{\mathrm{O}}^{\bullet \bullet }$): (V_app > 0) At positive voltages ${\mathrm{V}}_{\mathrm{O}}^{\bullet \bullet }$ formation and migration occurs. The negatively charged oxygen migrates towards the interface and into the porous Pt electrode and the positively charged ${\mathrm{V}}_{\mathrm{O}}^{\bullet \bullet }$ move along the electric field away from the Pt electrode and towards the grounded Ti electrode. (V_app ≤ 0) At zero applied voltage, ${\mathrm{V}}_{\mathrm{O}}^{\bullet \bullet }$'s move back towards the Pt, driven by the built-in electrochemical gradient, where they get filled by oxygen. A negative voltage accelerates this process. c Conductance change from low to high under the application of 100 SET pulses with an amplitude of 4V and a duration of 500 μs. d Time-dependent conductance measurement (read out at 0.6V) when voltage pulses (2V, 2.5V, and 3V) with a duration of 100μs are applied. The pulses induce short-term increases of the conductance with subsequent decay. The long-term conductance (red area) remains constant. e Short-term conductance changes due to the voltage pulses in the dotted rectangle of (d). Only the conductance values during the read voltage are shown here. f Aggregate plot showing short-term plasticity for different values of the long-term weight W. The measurement data was obtained by first applying the protocol in (d) to characterize the short-term plasticity for the minimum long-term weight (W₁). The long-term weight was then changed by 100 SET pulses (c) and, after a waiting period of 240s, the short-term plasticity was measured again.

Fig. 3. Control over the magnitude and dynamics of short-term conductance updates.

a Mean (solid line) and standard deviation (shaded area) of pulse-induced short-term conductance updates (ΔF) from five conductance measurements and using four different pulse voltages (2, 2.5, 3, and 3.5V). The read voltage is set to 0.6V and the pulse width to 100 μs. To better compare the measurements, the conductance values were adjusted by subtracting the initial conductance at t=0 from the data. b Heatmap of the achieved ΔF for the different pulse voltages and widths. c Heatmap of the required pulse energy for the same voltage and width combinations as in (b). d (Top) Applied voltage protocol on a linear x-axis using 0.6V/200 μs read pulses with a period of 700 μs and (bottom) corresponding conductance values shown on a logarithmic x-axis. In between the read pulses a constant bias voltage (V_bias) of variable amplitude is applied. The main pulse voltage and width are set to 3.5V and 500 μs, respectively, in all measurements. The mean and standard deviation of the adjusted conductance values are shown for 5 measurements on a semi-log plot. e Extracted decay time constant Λ from the measurements in d) as a function of V_bias. The experimental data points were fitted with a sigmoid function.

Fig. 4. Simulation and energy consumption of an STPN network with multi-functional synapses.

a Sketch of the full STPN network. A frame of the Atari game is fed into two convolutional layers: Conv(kernel, stride) plus a ReLU activation function. The features are then fed into the m-STPN layer (blue ellipses and lines). The layer’s output is split into actions and a value by two fully connected linear layers. b, c Average reward as a function of agent steps during training for (b) three different implementations of the STPN layer and (c) five different ranges of Λ. Each curve represents the average reward of 16 agents with different random parameter initialization. The shaded area denotes the standard deviation. In the inset of c), the cases Λ = 0 and Λ = [0.08, 0.92] (i.e., the achievable device range) are shown. d Total synaptic weight (long- and short-term component) of a single synapse of the trained network (S_max{ΔF}) during an entire game. The Zoom-in additionally shows the long-term weight W in red and the ΔF as black bars. e Energy consumed by our memristors due to ΔF updates, i.e., voltage pulses with widths w_p, fitted by a power-law. f Power consumed by our memristors due to different Decay bias voltages (V_bias). g Time evolution of the energy consumption of the synapse in (d) during an entire Pong game for a memristor (blue) and a pure GPU implementation (orange). Different energy contributions and the total energy are shown. h Histograms of all synapses in the network, indicating how many synapses consume a specific amount of energy during the whole Pong game. The two contributions ΔF and Decay are shown. For the Decay the worst case scenario: V_bias = 0.6 is assumed for all synapses. i Total energy histogram (ΔF plus Decay).