Thalamo-cortical spiking model of incremental learning combining perception, context and NREM-sleep

Abstract

The brain exhibits capabilities of fast incremental learning from few noisy examples, as well as the ability to associate similar memories in autonomously-created categories and to combine contextual hints with sensory perceptions. Together with sleep, these mechanisms are thought to be key components of many high-level cognitive functions. Yet, little is known about the underlying processes and the specific roles of different brain states. In this work, we exploited the combination of context and perception in a thalamo-cortical model based on a soft winner-take-all circuit of excitatory and inhibitory spiking neurons. After calibrating this model to express awake and deep-sleep states with features comparable with biological measures, we demonstrate the model capability of fast incremental learning from few examples, its resilience when proposed with noisy perceptions and contextual signals, and an improvement in visual classification after sleep due to induced synaptic homeostasis and association of similar memories.

Fig 1. Thalamo-cortical spiking model (ThaCo).

Panels B and C to compare the architecture of our model with the biological principles described in [1]. A) Scheme of the Thalamo-cortical spiking model (ThaCo). Input images, passed through a filter (HOG) are projected (blue arrow) to thalamic excitatory neurons (tc), mimicking the mechanism of the retinal visual stimulus. Thalamic neurons stimulate cortical excitatory neurons (cx) with a perceptual feedforward excitation (blue). Cortico-cortical and cortico-thalamic are considered as top-down prediction connections (red). (Red arrows—context/prediction) Currents coding for higher abstraction features incoming from other cortical areas. Cortical inhibitory neurons (in) arbitrate competition among cortical groups in a soft Winner-Take-All mechanism (WTA). Inhibitory reticular neurons (re) control the thalamic firing rate. The cortical layer is in turn connected to readout neurons (ro) B) A cellular mechanism for associating feed-forward and feedback signals. Low-level features are encoded in primary sensory regions and this signal propagates up the visual hierarchy (e.g. striate cortex (V1) sensitive to orientation, V4 sensitive to colour, V5 sensitive to motion, and inferior temporal (IT) cortex sensitive to shapes and objects). Higher-level areas provide feedback information (context or expectation) to lower areas. The ThaCo model presented in this paper is a single area model and the contextual signal is assumed to collect during training the knowledge carried by all other areas in the hierarchy (see red arrows in panel A). C) Conceptual representation of the back-propagation activated calcium (BAC) firing hypothesis supporting efficient binding of features and recognition. Pyramidal neurons receiving predominantly feed-forward information are likely to fire steadily at low rates, whereas the simultaneous presence of contextual and perceptual streams changes the mode of firing to bursts (BAC firing). This coincidence mechanism is mimicked in our ThaCo model. D) During training (left), the injection of contextual signal, plays the role of internal prediction and increases the perceptual threshold of a subset of cortical neurons. The simultaneous presence of perceptual and contextual promotes a high firing rate in such neurons, mimicking the BAC mechanism. Also, the simulataneous presence of the signal from the cortical layer and of the contextual signal promotes a high firing rate in readout neurons. In the classification phase (centre) the contextual signal is turned off. In the sleeping phase (right), the sensory pathways are turned off, and all the activity is generated spontaneously.

Fig 2. Sleep-like features.

A) State-wise differences of average firing rate, to be compared with Fig 2A by Watson et al. [25]. Cumulative distribution of the firing rates of individual cortical neurons (log scale); note the brain-state dependent differences (colour). Vertical lines separate neurons sorted by AWAKE firing rates into six subgroups (sextiles) with an equal number of elements. B) Firing rate changes across sleep in each of the six groups defined by the awake firing rates, to be compared with Fig 3B by Watson et al. [25]. High firing rate neurons show decreasing activity; low firing rate neurons do not increase their activity over sleep. C) Opposite modulation of neurons of different firing rates, to be compared with Fig 3D by Watson et al. in [25]. Comparison of individual neuron firing rates during the first and last packet of sleep. The regression line is significantly different from unity, showing that high and low firing rate neurons are oppositely modulated over sleep. D) Cortical neuron population mean firing rate changes across sleep, to be compared with Fig 3B by Watson et al. in [25]. E) Awake firing rate distribution of cortical neurons pre-sleep (upper plot) and plot-sleep (lower plot). Solid lines depict descriptive statistics parameters: Q1, 25% quartile; Q2, 50% quartile (median); Q3, 75% quartile. Middle plot: boxplots of the distributions. The central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively.

Fig 3. Incremental learning with alternation of awake training and sleep in ThaCo.

A) Dots: incoming current to each cortical neuron versus its adaptation current during different network stages of the activity presented in B. Pre-sleep classification phase (represented in blue, number 1), early sleep-like phase (in red, number 2), late sleep-like phase (in orange, number 3) and post-sleep classification phase (in green, number 4). Ellipses: Areas in the plot associated with different network stages, estimated from data through a Gaussian Mixture Model with a full covariance matrix. During sleep, the total input current to the cortical neurons decreases due to the sleep-induced homeostatic effect, that reduces recurrent connections weights in the cortical layer (see the transition from number 2 to number 3 in the diagram), notwithstanding the constant external aspecific stimulus. Sleep-like activity, on the other hand, affects the network status during the following awake classification phase 4: the effect of STDP during sleep is a general reduction and homogenization of input current distribution, as shown in a comparison between the pre-sleep stage 1 and post-sleep stage 4 in the diagram. B and C) Spiking cortical and thalamic activity produced during training (10 examples, one per digit class), classification (20 images) and sleeping phase for two consecutive sets respectively. First row: mean firing rate of the cortical neurons trained over a set of 10 examples (each set is used to independently train 200 cortical neurons, 20 per digit example); during the sleeping phase, the slow oscillation frequency trend in time is also depicted. Second row: raster plot of the first 400 cortical neurons. Third row: mean firing rates of thalamic neurons. Once recruited in the training phases, the cortical neurons participate in classification and sleeping phases.

Fig 4. Comparison of the average accuracy of the proposed thalamo-cortical spiking model compared to artificial K-nearest-neighbour incremental algorithms in absence of noise (solid lines) and with noisy inputs(dotted lines).

A) We infer the network answer to the classification task in two different ways: 1) Digit-Class Readout, as the class associated to the most active subgroup of readout neurons in ThaCo (supervised approach); 2) Example-Specific Group, by mapping the class over which the most active subgroup of cortical neurons has been trained (unsupervised approach). Solid lines depict accuracy in absence of noise, dotted lines depict accuracy with “Salt and Pepper” noise (density = 0.2) injected into the unprocessed MNIST images for both training and classification phases. Accuracy is assessed on an independent test set, consisting of 500 examples; the average and standard error of the mean (SEM) are evaluated on 20 different trials using independent training sets in each. B), C) represent the same plots shown in A) on different scales, for visualization purposes and for highlighting selected features. Specifically, B) ThaCo behaves like Knn-k1 for a small number of examples; C) as the number of training examples increases, ThaCo Digit-Class-Readout exhibits performances that are comparable with higher-order Knn algorithms.

Fig 5

A) Sleep mitigates the effects of noisy contextual signals on the classification performances of ThaCo Classification performances evaluated over the Readout layer (supervised learning protocol, left) and Cortical layer (unsupervised learning protocol, right). Comparison among the network trained over non noisy examples (blue), the network trained over noisy examples without any deep-sleep like phase (violet), and the network trained over noisy examples interposing a deep-sleep-like activity between the training and the classification phases (green). The contextual signal provided in the training phase is corrupted by noise (i.e. some examples are associated with stronger synapses), leading to a drop in performances (comparison between blue and violet line). Still, the interposition deep-sleep-like phases between noisy-training and classification phases recovers the performances of the network trained with a non-noisy protocol. B) Sleep-induced homeostatic and associative effects on cortico-cortical synaptic-weight distributions. Pre-sleep (violet), post-sleep (green). Solid lines: mean and standard deviation. a) Intra-group connections: weight distributions of synapses connecting neurons trained over the same example (i.e. that during the training stage were triggered by the same contextual stimulus, thus activated simultaneously during a specific training example); b) Intra-class connections: weight distributions of synapses connecting neurons trained over different examples belonging to the same class (i.e. that have not been simultaneously triggered by the contextual stimulus in the training phase, but still have been triggered by a sensorial thalamic signal associated to images belonging to the same class) c) Inter-class connections: Connections among groups trained over different classes (i.e. triggered by the contextual stimulus together with a sensorial thalamic signal associated to images belonging to different classes). We note the homeostatic effect of sleep (in A) leading to a general reduction of weights associated to example-specific synapses and a reinforcement of the intra-class connections (in B). Synapses connecting groups trained on different examples, on the other hand, are much less affected by sleep. The inset, showing part of the same plot in a lin-lin scale, is added to illustrate the shape of the pre-sleep distribution, and the difference in the mean values before and after sleep.

Fig 6. Effects of sleep on intra-class and example-specific synapses after training with a noisy contextual signal.

Comparison of synaptic-weight matrices, pre-sleep (Left) vs post-sleep (Right). Training over 5 examples per class (20 neurons per example). A) and B) depict all cortico-cortical synaptic weights connecting the full set of trained neurons (colour bar, logarithm scale), black lines separate neurons solicited by contextual signal together with thalamic sensorial signal pointing to images belonging to the same digit class in the training phase; C) and D) focus on the synaptic weights connecting groups of cortical neurons simultaneously solicited by a contextual signal in the training phase (thus stimulated over the same sensorial signal identifying images belonging to the same class) (colour bar, linear scale), vertical black lines separate neurons trained over different categories, horizontal black lines separate cortical groups of neurons solicited during the presentation of the same 10 examples (one per digit class). The post-sleep intra-class cooperation is evident in B and in agreement with Fig 5B.b, while the homeostatic effect over example-specific synapses is manifest in D as already suggested by Fig 5B.a. The strong, example-specific differences in synaptic weights (e.g. third example of class 4) are due to the noisy training protocol that introduces randomness in the magnitude of the contextual signal that reaches the example-specific group.

Fig 7. Training and classification phases.

A) Training Phase: the sensorial perception (blue arrow) is encoded into the thalamic excitatory neurons. A time-specific contextual signal is delivered to a subset of cortical excitatory neurons (red arrow), raising their perceptual threshold on the example-specific thalamic activity and inducing in such group of neurons a high firing rate during training (represented with red flames drawing). STDP induces group-specific connectivity in the subset of facilitated cortical neurons, and the thalamic pattern is sculptured into synapses that connect the thalamus with the example-specific group. The readout layer, made of as many groups of cortical neurons as the number of classification classes, is also trained through the simultaneous administration of a class-specific contextual signal addressing the subset associated to the correct class label (red arrow). B) Classification Phase: no contextual signal is given to the cortex. One or more subgroups of excitatory cortical neurons reach a high (red flames), intermediate (yellow flames) or low (no flames) level of activity, depending on the similarity between the stimulating thalamic pattern and the training set. Here, the WTA mechanism is essential to decide the classification answer; the network decision can be evaluated measuring the activation level of the groups, either in the cortex or in the readout layer. C) Combination of contextual and perceptual signals to create one group of cortical neurons sensible to a specific example in a soft winner-take-all mechanism. Three examples are presented to three cortical groups for 2s (start and stop presentation time marked by green and red dashed lines). High firing rate is induced only when the example-specific cortical group is reached by both the thalamic (perceptual) stimulus and the contextual (example-specific) signal. Upper row: mean firing rates of a cortical subgroup of cortical neurons; lower row: mean membrane potentials (the black dotted line depicts the firing threshold potential). Left column: Neuron activity when stimulated by the thalamic signal only (perception): null firing rate and under-threshold membrane potential. Central column: Neuron activity when stimulated by contextual signal only (internal prediction): a moderate firing rate is induced. Right column: Simultaneous perceptual and contextual signals induce a high firing rate in the example-specific group. D) Soft Winner-take-all dynamics among example-specific groups of cortical neurons during retrieval and classification phases. Mean firing rates of three groups trained to be sensitive to three different examples. D-Rows) Firing rates of the firs (blue), second(orange), third (green) neural group. D-Retrieval phase column) Exactly the three learnt examples (belonging to three different digit classes) are re-presented to the network without any contextual signal, resulting in an almost hard-WTA dynamics among the three groups. D-Classification phase column) Three novel images, for which the network has not been trained, are presented to the network without any hint from the contextual signal; a soft-WTA dynamics is emerging, rewarding for each presentation the neuron group with the highest firing rate and still allowing all the other groups to fire with non-zero probability. Here read-out neurons are not represented and the figure demonstrates how it is possible to extract a classification answer looking at the cortical layer only.