Adaptive control of synaptic plasticity integrates micro- and macroscopic network function

Abstract

Synaptic plasticity configures interactions between neurons and is therefore likely to be a primary driver of behavioral learning and development. How this microscopic-macroscopic interaction occurs is poorly understood, as researchers frequently examine models within particular ranges of abstraction and scale. Computational neuroscience and machine learning models offer theoretically powerful analyses of plasticity in neural networks, but results are often siloed and only coarsely linked to biology. In this review, we examine connections between these areas, asking how network computations change as a function of diverse features of plasticity and vice versa. We review how plasticity can be controlled at synapses by calcium dynamics and neuromodulatory signals, the manifestation of these changes in networks, and their impacts in specialized circuits. We conclude that metaplasticity-defined broadly as the adaptive control of plasticity-forges connections across scales by governing what groups of synapses can and can't learn about, when, and to what ends. The metaplasticity we discuss acts by co-opting Hebbian mechanisms, shifting network properties, and routing activity within and across brain systems. Asking how these operations can go awry should also be useful for understanding pathology, which we address in the context of autism, schizophrenia and Parkinson's disease.

Fig. 1. Common elements of synaptic plasticity.

a A pyramidal neuron with an afferent axon impinging on a dendrite, with two probes shown for stimulation and/or recording. b Three example stimulation protocols. Top: Burst inducing stimulation of an afferent connection (teal). Middle: STDP protocol, afferent stimulation (teal) paired with subsequent soma stimulation (purple). Bottom: A spike-burst protocol. Many other protocols exist, e.g. using different current injections, repetition timings and numbers, etc. c Phosphorylation of AMPARs changes their membrane densities and relative compositions of GluA1 to GluA2 subunits (shown in red and blue), mediating EPSC amplitudes. Similar changes in NR2A vs NR2B (as opposed to NR1) subunits of NMDARs modify their relative calcium permeability. Colors are visual guides. d Plasticity vs. Ca²⁺ concentration in canonical metaplasticity. A small amount of Ca²⁺ entering the postsynaptic cell induces LTD; larger amounts induce LTP. Changes in Ca²⁺ permeability change the amount of calcium delivered for a given depolarization, acting like a changeable ("floating") threshold, enforcing homeostasis, and inducing competitive learning. More realistic models are more sophisticated (see e.g., [56]) but this is a well established and reasonable first approximation.

Fig. 2. Fundamentals of Hebbian plasticity in neural networks.

Blue indicates less activity or reduction in efficacy, and red indicates the opposite throughout. a A feed-forward network with two stimulus presentations, "trial 1" and "trial 2". Neurons 1-4 are connected to 5 and 6, and connections to 5 are shown with arrows. Synapses are strengthened from neurons with higher than average activity (red) and weakened from those with weaker activity (blue). b The synapse updates arranged in an array, numbered according to neurons as (To,From). The principal component of the trial-by-trial variability in a is shown, also labelled by neuron. This vector is the same as the top row of the weight updates, illustrating how the update is a "matched filter" (parallel vector) for the PC. The response of unit 5 is the sum over all pairs of elements multiplied together, i.e. the sum of the weight number (5,1) times the activity in unit 1, the weight (5,2) times the activity in unit 2, etc. c RPE-modulated feature detectors extract features from subsets of the data, picking up the covariance of input-output transformations with reward, or generating matched filters for PCs that drive rewarding output activity. Such updates can perform reinforcement learning. d Recurrent network with two stimulus presentations, and weight changes under symmetric and asymmetric update rules (assuming neuron 2 fires before neuron 1, in asymmetric case). e Weight matrix updates, under the same convention as in (b). Grey diagonal elements indicate that neurons don't innervate themselves. Grey off diagonal elements show unchanged weights. Symmetric and antisymmetric updates are generated by rate-coded Hebbian, and causal STDP rules. Nilpotent updates can occur when an anti-symmetric update can't lower a weight any further. These are linked to so called "non-normal" dynamics, of which transient synfire chains are an example. f Activity propagation from unit 1 (with time represented as evolving downward) if weights were as shown in (e) (and neurons turn themselves off). In the symmetric case, 1 and 2 are mutually excitatory, passing spikes back and forth. In the latter cases, 1 excites 2, but is not reciprocally excited, so activity is transient.

Fig. 3. Neuromodulated plasticity and STDP kernels.

A STDP is established empirically by inducing pre- and post-synaptic spiking and observing synapse efficacy changes. This determines a "kernel" or window function, which can be used to model plasticity. (a-i) Kernel components can theoretically be mixed and matched, and in-vivo they appear to be. The vertical axis is change in synaptic efficacy, horizontal is time from presynaptic to post-synaptic spike. The canonical STDP kernel is in (c), but unipolar LTP (a), unipolar LTD (i) and anti-causal STDP (g) also appear under conditions of neuromodulation. The NE, ACh and DA-dependent plasticity discussed in text can often be regarded as specifying these kernels. Hippocampal DA appears to convert canonical STDP (c) to unipolar LTP (a), whereas hippocampal ACh appears to convert canonical STDP (c) to unipolar LTD (i) in some circumstances, for example. In some PFC synapses, DA gates unipolar potentiation (a). Cases (d), (b), (f), and (h) also seem to arise in other conditions. See [62] for a detailed review, and references in text. Note that any particular effect may be protocol dependent. B Illustration of connectivity effects. Organizing synaptic efficacies as a matrix, updates based on different kernels have different mathematical properties. Unipolar LTP or LTD are both symmetric, classic STDP and inverted STDP are both anti-symmetric, and the remaining possibilities, are called nilpotent. Symmetric matrices have real eigendecompositions, meaning roughly that networks with symmetric connections produce stable recurrent activity. Anti-symmetric network weights and nilpotent weights favor transient, "moving" activity, such as synfire chains. Because activities and weights in actual networks are rectified, asymmetric updates will often be rectified to nilpotent ones, and thereby push networks to have feed-forward sub-networks. Here, a network of three neurons with positive symmetric initial weights between neurons 2 and 3 undergoes asymmetric updates based on the classic asymmetric STDP kernel to produce unidirecitonal connectivity from neuron 2 to 3. C Example application of the REINFORCE algorithm, specifying a three-factor plasticity rule, to a two neuron network. On repeated trials, activity in unit 2 dictates network performance such that being closer than average to some hypothetical target T is rewarded, being further is punished. These outcomes generate reward prediction errors relative to average reward both of which serve to increase the weight from unit 1 to unit 2 under the equation for dW, the change in synapse strength.

Fig. 4. An integrated view of dopamine modulated learning in the striatum.

a Striatal medium spiny neuron plasticity and excitability depend on DA, ACh, adenosine, and NMDARs. Balance between the two pathways is mediated by DA. Figure follows Shen et al. 2008. b Cortico-basal-ganglia loops connect the cortex to the striatum recurrently and hierarchically. Figure follows Obeso et al 2014. c Internal detail of a single loop, showing dual pathways from striatal D1 and D2-expressing neurons to different output structures along the direct and indirect pathways. D1 MSNs largely reside in the "Go" pathway, D2 MSNs in the "NoGo" pathway. GP denotes globus pallidus. d The Opponent Actor Learning model, which captures cortico-striatal contributions to reinforcement learning computationally. See text for details. e When dopamine is high, the biological details of the model recapitulate empirically observed enhancements in learning to pick the best among good options. When DA is low, the model recapitulates enhanced ability to avoid bad options. These effects arise from the Hebbian reward prediction error modulated strengthening of Go and No-Go pathway MSNs. ΔG = GE ΔN = NE E f Pathological feedback of low DA on enhanced No-Go learning.