A role for cortical interneurons as adversarial discriminators

Abstract

The brain learns representations of sensory information from experience, but the algorithms by which it does so remain unknown. One popular theory formalizes representations as inferred factors in a generative model of sensory stimuli, meaning that learning must improve this generative model and inference procedure. This framework underlies many classic computational theories of sensory learning, such as Boltzmann machines, the Wake/Sleep algorithm, and a more recent proposal that the brain learns with an adversarial algorithm that compares waking and dreaming activity. However, in order for such theories to provide insights into the cellular mechanisms of sensory learning, they must be first linked to the cell types in the brain that mediate them. In this study, we examine whether a subtype of cortical interneurons might mediate sensory learning by serving as discriminators, a crucial component in an adversarial algorithm for representation learning. We describe how such interneurons would be characterized by a plasticity rule that switches from Hebbian plasticity during waking states to anti-Hebbian plasticity in dreaming states. Evaluating the computational advantages and disadvantages of this algorithm, we find that it excels at learning representations in networks with recurrent connections but scales poorly with network size. This limitation can be partially addressed if the network also oscillates between evoked activity and generative samples on faster timescales. Consequently, we propose that an adversarial algorithm with interneurons as discriminators is a plausible and testable strategy for sensory learning in biological systems.

Fig 1. Learning representations with a discriminating interneuron.

a) Representations are defined by a generative model that relates them to inputs. Organisms must learn to infer these representations from inputs (for example, to infer a visual category). b) This computational problem is equivalent to matching the joint probability distribution over inputs and representations during inference and generation. c) The brain may solve this problem with neurons that act as adversarial discriminators (red) of whether activity in neighboring neurons (blue) is dreamed or is stimulus-evoked. To achieve this, their input synapses would switch from a Hebbian to anti-Hebbian plasticity with the switch from wake to sleep.

Fig 2. A simple case illustrating the problem of predicting activity in a population with internal connections.

a) We created a toy task in which the goal is to adjust the connections away from neuron A to better model the activity of neurons B and C. The “target population” [B, C] is also driven by an external teaching signal from neuron T. All connections evoke Gaussian-distributed postsynaptic responses, and all neurons sum their inputs linearly. b) When learning the A → B and A → C connections by predicting the postsynaptic neuron (e.g. maximizing log p(B|A), ignoring recurrence), the predictions fail to match the joint distribution. The left plot shows the true joint distribution p(B, C), which is correlated and non-Gaussian due to the B → C connection internal to the target population, while the right plot shows ${\mathbb{E}}_A\left[p(B,C|A)\right]$. c) The adversarial strategy successfully aligns the distribution. The discriminator sees B and C and gates plasticity at the predictive connections from A. d) The alignment can be quantified with the KL divergence of the binned histogram between the learned and target distribution. We plot the trajectories from 5 random runs; all networks share the same initialization. e) Another measure of success is the distance of the parameters of the outward connections from A from their optimal value, which we call the parameter error.

Fig 3. Approximate inference in a generative model of MNIST digits using a recurrent, stochastic autoencoder.

a) In both inference and generation, samples are processed through a multilayer network that parameterizes a multivariate Gaussian. The network is then sampled and undergoes nonlinear stochastic recurrence. b) Performance metric: Following Berkes et al. [3], we empirically verify if the joint distributions of ‘sleep’ and ‘wake’ phases match by comparing the frequency over a binned histogram. We quantize the vectors of x and z by rounding to the nearest integer and quantify the histograms of the frequency of each pattern across 32 units’ activity when observing/generating MNIST digits. Perfect performance corresponds to all points lying on the y = x diagonal. c) As a baseline, we train the architecture with the standard variational autoencoder objective function (a.k.a. the ELBO), approximating the inferred distribution as a Gaussian and essentially ignoring the stochastic recurrence. This approach unsurprisingly fails, with patterns in ‘sleep’ occurring at different frequencies than in ‘wake’. d) In the adversarial wake/sleep algorithm, a global discriminator observes both z and x during wake and sleep phases. e) This algorithm enables realistic generation but often results in poor reconstruction. f) The frequency of patterns over a subsample of units is much more similar in sleep and wake than in (c). g) The oscillatory algorithm: During the wake phase, inputs are reconstructed through the generative model, and the discriminator attempts to classify real from reconstructed. The discriminator also aims to classify wake from sleep. h) Reconstruction improves due to oscillations without negatively affecting generation. i) Matched patterns for the oscillatory algorithm.

Fig 4. An experimental program to test the adversarial hypothesis.

a) A key first step is to test whether the brain’s learning algorithms actually do align the distribution of activity in wake and sleep. This would begin with identifying neurons in chronic recordings that have the same distribution of activity patterns in wake and sleep. A subset of these neurons could then be perturbed during sleep to produce a new distribution (pink). Over the course of several nights, it should be the the case that the distribution of stimulus-evoked activity should align with the perturbed distribution. This experiment could identically be run by perturbing the waking activity and observing learning in sleep. b) A next step is to identify if this alignment is mediated by discriminating interneurons. This might begin with a survey of gene transcription in interneurons in wake and sleep, with the prediction that some class should exclusively transcribe genes associated with postsynaptic long-term depression (LTD) in sleep, and genes associated with postsynaptic long-term potentiation (LTP) in waking states. Blocking this synaptic plasticity in only this cell type should prevent the alignment observed in the experiment in (a), even though it concerns different cell types.