Input correlations impede suppression of chaos and learning in balanced firing-rate networks

Abstract

Neural circuits exhibit complex activity patterns, both spontaneously and evoked by external stimuli. Information encoding and learning in neural circuits depend on how well time-varying stimuli can control spontaneous network activity. We show that in firing-rate networks in the balanced state, external control of recurrent dynamics, i.e., the suppression of internally-generated chaotic variability, strongly depends on correlations in the input. A distinctive feature of balanced networks is that, because common external input is dynamically canceled by recurrent feedback, it is far more difficult to suppress chaos with common input into each neuron than through independent input. To study this phenomenon, we develop a non-stationary dynamic mean-field theory for driven networks. The theory explains how the activity statistics and the largest Lyapunov exponent depend on the frequency and amplitude of the input, recurrent coupling strength, and network size, for both common and independent input. We further show that uncorrelated inputs facilitate learning in balanced networks.

Fig 1. Suppression of chaos in balanced networks with common vs independent input.

A) Common input: External input $I_i^{\text{in}}\left(t\right)=\sqrt{N}{I}_0+\delta I\left(t\right)$ consist of a positive static input and a sinusoidally time-varying input with identical phase across neurons. B) Independent input: External input $I_i^{\text{in}}\left(t\right)=\sqrt{N}{I}_0+\delta I_i\left(t\right)$ consist of a positive static input and a sinusoidally time-varying input with a random phase for each neuron. C) External inputs (top), recurrent feedback $I_i^{\text{rec}}={\sum }_j{J}_{ij}\varphi \left(h_j\right)$ and their population average (thick line) (middle), and synaptic currents (bottom) for three example neurons. Recurrent feedback has a strong time-varying component that is anti-correlated with the external input, resulting in cancellation. D) Same as in C, but for independent input. Here, no cancellation occurs and the network is entrained into a forced limit cycle. Throughout this work, green (violet) refers to common (independent) input. Model parameters: N = 5000, g = 2, f = 0.01/τ, I₀ = J₀ = 1, I₁ = 6.

Fig 2. Largest Lyapunov exponent shows different chaos suppression for common vs independent input.

Largest Lyapunov exponent λ₁ as a function of input modulation amplitude I₁ for common (green) and independent (violet) input. $I_1^{\text{crit}}$ are the zero-crossings of λ₁ and thus the minimum I₁ required to suppress chaotic dynamics. With common input, λ₁ crosses zero at a much larger I₁. Dots with error bars are numerical simulations, dashed lines are largest Lyapunov exponents computed by dynamic mean-field theory (DMFT). Error bars indicate ±2 std across 10 network realizations. Model parameters: N = 5000, g = 2, f = 0.2/τ, I₀ = J₀ = 1.

Fig 3. Difference in chaos suppression increases with network size, tightness of balance, and near the transition to chaos.

A) Dependence of $I_1^{\text{crit}}$ on network size N. With common input, $I_1^{\text{crit}}\propto \sqrt{N}$ for large N, but is constant for independent input. Error bars indicate interquartile range around the median. B) Dependence of $I_1^{\text{crit}}$ on ‘tightness of balance’ parameter K, which scales both I₀ and J₀. Results for large K are the same as in A but for small K, the network is no longer in the balanced regime, and results for common and independent input become similar. Error bars indicate ±2 std. C) Dependence of $I_1^{\text{crit}}$ on gain parameter g for low input frequency f. Close to $g_{\text{crit}}$, an arbitrarily small independent input can suppress chaos; this is not the case with common input. The quasi-static approximation (dotted) and DMFT (dashed) results coincide. Error bars indicate ±2 std. Model parameters: I₀ = J₀ = 1 in A and C; g = 2, f = 0.2/τ in A and B; $I_0=J_0=\sqrt{K/N}$, in B; f = 0.01/τ in C, N = 5000 in B and C.

Fig 4. Mechanism of chaos suppression with slowly varying common input.

A) External input $I_i^{\text{in}}\left(t\right)=\sqrt{N}{I}_0+\delta I_i\left(t\right)$ (dashed) and recurrent input $I_i^{\text{rec}}\left(t\right)={\sum }_j{J}_{ij}\varphi \left(h_j\left(t\right)\right)$ (solid) for three example neurons. B) Synaptic currents h_i for four example neurons. C) Local Lyapunov exponent from network simulation, which reflects the local exponential growth rates between nearby trajectories (solid), and Lyapunov exponent from stationary DMFT (dashed) used in quasi-static approximation. When $I_1>\sqrt{N}{I}_0$, external input periodically becomes negative and silences the recurrent activity (gray bars). During these silent episodes, the network is no longer chaotic and ${\lambda }_1^{\text{local}}=-1/\tau$. When the input is positive, dynamics remains chaotic and ${\lambda }_1^{\text{local}}>0$ on average. Model parameters: N = 5000, g = 2, f = 0.01/τ, I₀ = J₀ = 1.

Fig 5. Dynamic mean-field theory captures frequency-dependent effects on the suppression of chaos.

A) $I_1^{crit}$ as a function of input frequency f (g = 1.6 light color, g = 2 dark color). $I_1^{\text{crit}}$ has a minimum that is captured by the non-stationary DMFT (dashed green line) but not by the quasi-static approximation (dotted green line), which does not depend on frequency f. At high f, the low-pass filter effect of the leak term attenuates the external input modulation for both cases, thus resulting in a linearly increasing $I_1^{\text{crit}}$. B) Dependence of $I_1^{\text{crit}}$ on the gain parameter g for high input frequency (f = 0.2/τ), showing a monotonic increase. The non-stationary DMFT results are in good agreement with numerical simulations. For comparison, we include the result of the quasi-static approximation (dotted green line), which shows a more gradual dependence on g and applies only at low frequencies (see Fig 3). Error bars indicate ±2 std. Model parameters: N = 5000, g = 2, f = 0.2/τ, I₀ = J₀ = 1.

Fig 6. Difference in chaos suppression in sparsely-connected E-I network.

λ₁ as a function of I₁ for common and independent inputs, showing a monotonic decrease with I₁ and a larger zero-crossing for common input. This result is qualitatively similar to that obtained in the single population network with negative mean coupling (Fig 2). Error bars indicate ±2 std, lines are a guide for the eye. Increasing the excitatory efficacy α increases λ₁ for both common and independent input (α ∈ {0, 0.5, 0.7}). Model parameters (parameters defined as in [16] for constant input and W_I1 and W_E1 are the modulation amplitudes of the input to the excitatory and inhibitory population): N_E = N_I = 3500, K = 700, g = 1.6, $J_{EE}=g\alpha /\sqrt{K}$, $J_{EI}=-1.11g/\sqrt{K}$, $J_{IE}=g\alpha /\sqrt{K}$, $J_{II}=-g/\sqrt{K}$, $W_E=g\alpha \sqrt{K}$, $W_I=0.44g\sqrt{K}$, W_E1 = gαI₁, W_I1 = 0.44gI₁, f = 0.2/τ.

Fig 7. Common input impedes learning in balanced networks.

A) Schematic of the training setup. A ‘student network’ (S) is trained to autonomously generate the output $F^{\text{out}}\left(t\right)=sin\left(2\pi ft\right)$, by matching its recurrent inputs to those of a driven ‘teacher network’, whose weights are not changed during training. B) λ₁ in the teacher network as a function of I₁. C) Test error in the student network as a function of I₁. Critical input amplitude $I_1^{\text{crit}}$ is indicated by vertical dashed lines. Consistent with the difference in $I_1^{\text{crit}}$, the teacher networks driven with common input require a larger I₁ to achieve small test errors in the student network. Error bars indicate interquartile range around the median. D) Top: Target output $F^{\text{out}}$ (green) and actual output z (dashed orange) for two input amplitudes I₁ ∈ {5, 15}. Bottom: Firing rate ϕ(h_i) for two example neurons in teacher network with common input (green full line) and student network (orange dotted line) for two input amplitudes. E) Scatter plot of test error as a function of λ₁ for each network realization in A and B, with both common and independent input. When chaos in the teacher network is not suppressed (λ₁ > 0), test error is high. Training is successful (small test error) when targets are strong enough to suppress chaos in the teacher network. Training is terminated when error reaches below 10⁻². Model parameters: N = 500, g = 2, I₀ = J₀ = 1, ϕ(x) = max(x, 0) in both teacher and student networks; f = 0.2/τ in the teacher network inputs and target $F^{\text{out}}$.

Fig 8. Activity, population firing rate and autocorrelations of balanced networks with common input.

A) Firing rates ϕ_i(t) = ϕ(h_i(t)) of three example units. B) Mean population firing rate ν(t). C) Time-averaged two-time autocorrelation function (Eq 5) as a function of time difference with no external input (I₁ = 0). D-F) Same as A-C but for input amplitude $I_1=0.8\sqrt{N}\approx 56.5$; activity remains chaotic. G-I Same as A-C but for stronger input ($I_1=10\sqrt{N}\approx 707.1$); activity is entrained by the external input and is no longer chaotic. Dashed lines (middle and right columns) are results of non-stationary DMFT, full lines are median across 10 network realizations. Model parameters: N = 5000, g = 2, f = 0.05/τ, I₀ = J₀ = 1.

Fig 9. Activity, population firing rate and autocorrelations of balanced networks with independent input.

A) Firing rates ϕ_i(t) = ϕ(h_i(t)) of three example units. B) Mean population firing rate ν(t). C) Autocorrelation function with no external input (I₁ = 0). D)-F) Same as A-C but for input amplitude of I₁ = 0.8; activity remains chaotic. G)-I) Same as A-C but for stronger input (I₁ = 10); activity is fully controlled by the external input and is no longer chaotic. Dashed lines (middle and right columns) are results of stationary DMFT, full lines are median across 10 network realizations. Model parameters: N = 5000, g = 2, f = 0.05/τ, I₀ = J₀ = 1.

Fig 10. No qualitative difference in chaos suppression by common vs independent input in canonical random networks.

A) $I_1^{crit}$ as a function of input frequency f ($g=\sqrt{2}$ light color, g = 2 dark color). $I_1^{\text{crit}}$ has a minimum for both common and independent input. The independent input case is identical to the scenario studied in [3]. At high f, the low-pass filter effect of the leak term attenuates the external input for both cases, thus resulting in a linearly increasing $I_1^{\text{crit}}$. B) Dependence of $I_1^{\text{crit}}$ on the gain parameter g for both low input frequency (f = 0.01/τ, dark color) and high input frequency (f = 0.2/τ, light color), showing a monotonic increase. Error bars indicate ±2 std. Model parameters: N = 5000, $g\in \{\sqrt{2},2$}, f ∈ {0.01, 0.2}/τ, I₀ = J₀ = 0.