Correlated connectivity and the distribution of firing rates in the neocortex

Abstract

Two recent experimental observations pose a challenge to many cortical models. First, the activity in the auditory cortex is sparse, and firing rates can be described by a lognormal distribution. Second, the distribution of nonzero synaptic strengths between nearby cortical neurons can also be described by a lognormal distribution. Here we use a simple model of cortical activity to reconcile these observations. The model makes the experimentally testable prediction that synaptic efficacies onto a given cortical neuron are statistically correlated, i.e., it predicts that some neurons receive stronger synapses than other neurons. We propose a simple Hebb-like learning rule that gives rise to such correlations and yields both lognormal firing rates and synaptic efficacies. Our results represent a first step toward reconciling sparse activity and sparse connectivity in cortical networks.

Figure 1.

Lognormal distributions in cerebral cortex. A, B, Distribution of spontaneous firing rates in auditory cortex of unanesthetized rats follows a lognormal distribution (Hromádka et al., 2008). Measurements with the cell-attached method show that spontaneous firing rates in cortex vary within several orders of magnitude. The distribution is fit well by a lognormal distribution with some cells displaying firing rate above 30 Hz and an average firing rate of ∼3 Hz (black arrow). The error bars show 95% confidence intervals by bootstrapping. C, D, The distribution of synaptic weights for intracortical connections (Song et al., 2005). To assess this distribution, pairs of neurons in the network were chosen randomly, and the strength of the connections between them was measured using electrophysiological methods (Song et al., 2005). Most connections between pairs are of zero strength: the sparseness of cortical network is ∼20% even if the neuronal cell bodies are close to each other so that the cells have a potential to be connected (Stepanyants et al., 2002; Thomson and Lamy, 2007). This implies that, in ∼80% of such pairs, there is no direct synaptic connection. The distribution of nonzero synaptic efficacies is close to lognormal (Song et al., 2005), at least for the connectivity between neurons in layer V of rat visual cortex. This implies that the logarithm of the synaptic strength has a normal (Gaussian) distribution.

Figure 2.

Randomly connected white-noise network connectivity does not yield lognormal distribution of spontaneous firing rates. A, Synaptic connectivity matrix for 200 neurons. Because synaptic strengths are uncorrelated, the weight matrix looks like a white-noise matrix. B, Distribution of synaptic strengths is lognormal. The matrix is rescaled to yield a unit principal eigenvalue. C, Synaptic weights and firing rates of 12 randomly chosen neurons tended to be similar. Every circle corresponds to a single neuron, with diameter proportional to the spontaneous firing rate of the neuron. Thickness of connecting lines is proportional to strengths (synaptic weights) of incoming connections for each neuron. Red and blue circles and lines show spontaneous firing rates and incoming connection strengths for two neurons with maximum and minimum firing rates from the sample shown. Because incoming synaptic weights are similar on average, the spontaneous firing rates (circle diameters) tend to be similar. D, Spontaneous firing rates given by the components of principal eigenvector of matrix shown in A. The distribution of spontaneous firing rates is not lognormal, contrary to experimental findings (see Fig. 1A,B). The spontaneous firing rates are approximately the same for all neurons in the network.

Figure 3.

Correlated synaptic weights on the same axon (output correlations) do not lead to lognormal distribution of spontaneous firing rates. A, Synaptic weight matrix for 200 neurons contains vertical “stripes” indicating correlations between synapses made by the same presynaptic cell (the same axon). B, Distribution of synaptic weights is lognormal. C, Firing rates and synaptic weights tended to be similar for different neurons in the network, as illustrated on an example of 12 randomly chosen neurons. Red and blue circles show neurons with maximum and minimum firing rates (of the sample shown), with their corresponding incoming connections. D, Column matrix fails to yield broader distribution of spontaneous firing rates than the white noise matrix (see Fig. 2).

Figure 4.

Correlations among synaptic weights on the same dendrite (input correlations) lead to lognormal distribution of spontaneous firing rates. A, Synaptic connectivity matrix for 200 neurons. Note the horizontal stripes showing input correlations. B, Distribution of synaptic weights is set up to be lognormal. C, Inputs into two cells; red and blue are shown by the thickness of lines in this representation of the network. Because synaptic strengths are correlated for the same postsynaptic cell, the inputs into cells marked by blue and red are systematically different, leading to large differences in the firing rates. For the randomly chosen subset containing 12 neurons shown in this example, the spontaneous firing rates (circle diameter) vary widely because of large variance in the strength of incoming connections (line widths). D, Distribution of spontaneous firing rates is lognormal and has a large variance for row matrix.

Figure 5.

Multiplicative Hebbian learning rule leads to wide network connectivity and firing rate distributions. A, Synaptic connectivity matrix for 200 neurons resulting from 1000 iterations of multiplicative Hebbian learning rule. This matrix displays plaid structure (horizontal and vertical stripes) indicating both input and output correlations. This feature is similar to both column and row matrices introduced in previous sections. B, The adjacency matrix for the weight matrix shows the connections that are present (non-0; black) or missing (equal to 0; white). Adjacency matrix defined here is transposed compared with the standard definition in graph theory. The adjacency matrix is 20% sparse and is not symmetric, i.e., synaptic connections formed a directed graph. C, D, Distributions of synaptic weights resulting from the nonlinear Hebbian learning rule (C) and spontaneous firing rates (D) were approximately lognormal, i.e., appeared as normally distributed on logarithmic axis.

Figure 6.

Experimental predictions of this theory. A, The presence of row connectivity (Figs. 4–5), sufficient for generation of dual lognormal distributions, implies correlations between synaptic strengths on each dendrite (the diameter of the red circle). In addition, if the nonlinear Hebbian mechanism is involved in generation of these correlations, the synapses on the same axon are expected to be correlated (plaid connectivity; see Fig. 5). B, To reveal these correlations, the LASS was calculated for each dendrite. The distribution of these averages for individual dendrites (rows) from Figure 5 is shown by gray bars. The SD of this distribution is ∼0.64 in natural logarithm units. The black histogram shows LASS distribution after the synapses were “scrambled” randomly, with their identification with particular dendrites removed. This bootstrapping procedure (Hogg et al., 2005) builds a white-noise matrix with the same distribution of synaptic weights but much narrower distribution of bootstrapped LASS. C, Distribution of SDs (distribution widths) of LASS for many iterations of bootstrap (black bars). The widths were significantly lower than the width of the original LASS distribution (0.64; gray arrow). This feature is indicative of input correlations.

Figure 7.

The results of nonlinear multiplicative learning rule when inhibitory neurons are present in the network. A, The absolute values of the weight matrix display the same plaid correlations as in the network with excitatory neurons only (see Fig. 5A). B, The adjacency matrix contains inhibitory connections. The presence of nonzero connection is shown by black points (20% sparseness). Positions of the inhibitory neurons in the weight matrix are indicated by the vertical blue lines (15%). C, The distribution of absolute values of synaptic strengths resulting from nonlinear Hebbian learning rule is close to lognormal with small asymmetry. D, The spontaneous firing rates are widely distributed with the distribution that is approximately lognormal.