Shared input and recurrency in neural networks for metabolically efficient information transmission

Abstract

Shared input to a population of neurons induces noise correlations, which can decrease the information carried by a population activity. Inhibitory feedback in recurrent neural networks can reduce the noise correlations and thus increase the information carried by the population activity. However, the activity of inhibitory neurons is costly. This inhibitory feedback decreases the gain of the population. Thus, depolarization of its neurons requires stronger excitatory synaptic input, which is associated with higher ATP consumption. Given that the goal of neural populations is to transmit as much information as possible at minimal metabolic costs, it is unclear whether the increased information transmission reliability provided by inhibitory feedback compensates for the additional costs. We analyze this problem in a network of leaky integrate-and-fire neurons receiving correlated input. By maximizing mutual information with metabolic cost constraints, we show that there is an optimal strength of recurrent connections in the network, which maximizes the value of mutual information-per-cost. For higher values of input correlation, the mutual information-per-cost is higher for recurrent networks with inhibitory feedback compared to feedforward networks without any inhibitory neurons. Our results, therefore, show that the optimal synaptic strength of a recurrent network can be inferred from metabolically efficient coding arguments and that decorrelation of the input by inhibitory feedback compensates for the associated increased metabolic costs.

Fig 1. Inhibitory feedback decreases noise correlations.

A: Schematic illustration of the simulated neural network. Poisson neurons in the external population make random connections to neurons in the excitatory and inhibitory subpopulations. The connection probability P_ext ∈ [0.01, 1] is varied to achieve different levels of shared external input to the neurons. The neurons in the inhibitory (inh.) and excitatory (exc.) subpopulations make recurrent connections (exc. to exc., exc. to inh., inh. to inh., inh. to exc.) with probability P_rec = 0.2. The strength of those connections is parametrized by a_rec. B: Mean pairwise correlations between any two neurons in the exc. and inh. subpopulations plotted against the mean output of the network for different values of P_ext in a feedforward network (a_rec = 0 nS). Pairwise correlations are calculated from the number of spikes each neuron fires in a time window ΔT = 1 s across many trials of the simulation. The plot is vertically separated into two parts to also illustrate the smaller differences at lower values of P_ext. C: Mean pairwise correlations as in B, for different values of α (ratio of inhibitory-to-excitatory synaptic strength), a_rec = 0.01 nS. The black line represents the pairwise correlations in a feedforward network without any recurrent connections (a_rec = 0). D: Total current from recurrent synapses for different values of α, as in C. E-F: Same as in C-D, but with fixed α = 20 and different values of a_rec.

Fig 2. Fano factor of single neurons and of populations.

A-C: Mean Fano factor of individual neurons for different values of P_ext: 0.01 (A), 0.2 (B), 1 (C). The strength of the recurrent synapses (a_rec) is color-coded. The mean Fano factor increases with the strength of the recurrent synapses. D-F: Same as in A-C but for the Fano factor of the population activity. The points represent the population Fano factor obtained from the simulation, and the lines are a weighted 7th-degree polynomial, used only as a visual aid. For P_ext = 0.01, the increase in Fano factor of individual neurons (A) can have a stronger effect on the population Fano factor than decreasing the pairwise correlations, resulting in an increase of the population Fano factor with high values of a_rec (D). For higher values of P_ext, the pairwise correlations greatly increase the population Fano factor, which then decreases with increasing a_rec.

Fig 3. Metabolic cost of the network activity.

A-C: Cost at resting state (λ_ext = 0). A: Cost of the excitatory synaptic currents from the background input (Eq 35) and excitatory action potentials evoked by the background input. B: Cost of the action potentials (both excitatory and inhibitory) evoked by the background input. C: Total resting cost obtained by summing A and B. D: The total cost of the network activity is plotted against the output of the network (the total post-synaptic firing rate). Filled areas represent individual contributions of each cost component: cost of action potentials from the external population, cost of the excitatory synaptic currents, and cost of the post-synaptic (evoked) action potentials. As P_ext increases, the contribution of external action potentials to the overall cost decreases. With increasing a_rec, the contribution of excitatory synaptic currents increases. E: The cost of increasing the mean input by one action potential (w_AP, Eq 19) is significantly lower for higher P_ext. However, although the difference between P_ext = 0.01 and P_ext = 0.2 is approximately 10-fold, the difference between P_ext = 0.2 and P_ext = 1 is only approximately 2-fold, as the cost of the external population starts to contribute less to the overall cost.

Fig 4. Shared input decreases the gain and increases the individual Fano factor.

A-C: The input intensity λ_ext needed to evoke a given firing rate (x-axis) with different connection probabilities P_ext relative to the input intensity for P_ext = 0.01. A: a_rec = 0 nS, B: a_rec = 0.2 nS, C: a_rec = 1 nS. For higher P_ext, higher values of λ_ext are needed to achieve the same post-synaptic firing rates as with lower values of P_ext. This effect becomes more pronounced in stronger recurrent synapses (E-F). D-F: Gain of the network (Eq 20). A higher P_ext leads to a lower gain of the population activity. G-I: Higher values of P_ext also increase the Fano factor of individual neurons.

Fig 5. Information transmission with cost constraints.

A: Information-metabolic efficiency E (Eq 10) for different values of recurrence strength a_rec. P_ext is color-coded. B: Contour plot of the information-metabolic efficiency. Contours are at 0.75, 1.0, 1.25, 1.5, 1.75, 2.0, and 2.25 bits/s. C-H: Contour plots showing the capacity-cost function C(W) (Eq 9) with dependence on the recurrence strength a_rec for different values of P_ext. The contours show the maximal capacities constraint at different values of W (see Table 1 for the costs and capacity values at the contours). The heatmaps in B-H were calculated using piece-wise cubic 2D interpolation (SciPy interpolator CloughTocher2DInterpolator [34]) from the grid calculated with P_ext values 0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.5, 0.8, 1 and a_rec values 0, 0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3, 0.5, and 1 nS.

Fig 6. Information-metabolic efficiency with multi-dimensional output.

A: Information-metabolic efficiency E (Eq 10) for different values of recurrence strength a_rec. P_ext is color-coded. B: Contour plot of the information-metabolic efficiency. Contours are at 1, 1.25, 1.5, 1.75, 2, 2.25, 2.5, and 2.75 bits/s.