Robust information propagation through noisy neural circuits

Abstract

Sensory neurons give highly variable responses to stimulation, which can limit the amount of stimulus information available to downstream circuits. Much work has investigated the factors that affect the amount of information encoded in these population responses, leading to insights about the role of covariability among neurons, tuning curve shape, etc. However, the informativeness of neural responses is not the only relevant feature of population codes; of potentially equal importance is how robustly that information propagates to downstream structures. For instance, to quantify the retina's performance, one must consider not only the informativeness of the optic nerve responses, but also the amount of information that survives the spike-generating nonlinearity and noise corruption in the next stage of processing, the lateral geniculate nucleus. Our study identifies the set of covariance structures for the upstream cells that optimize the ability of information to propagate through noisy, nonlinear circuits. Within this optimal family are covariances with "differential correlations", which are known to reduce the information encoded in neural population activities. Thus, covariance structures that maximize information in neural population codes, and those that maximize the ability of this information to propagate, can be very different. Moreover, redundancy is neither necessary nor sufficient to make population codes robust against corruption by noise: redundant codes can be very fragile, and synergistic codes can-in some cases-optimize robustness against noise.

Fig 1. The information propagation problem.

This problem is illustrated with the visual periphery, but the information propagation problem is general: it arises whenever information is transmitted from one area to another, and also when information is combined to carry out computations. (A) The retina transmits information about visual stimuli, s, to the visual cortex. The information does not propagate directly from retina to cortex; it is transmitted via an intermediary structure, the lateral geniculate nucleus (LGN). Consequently, the information about the stimuli that is available to the cortex, denoted I_y(s), is not the same as the information that retina transmits, denoted I_x(s). Here, we ask what properties of neural activities in the periphery maximize the information that propagates to the deeper neural structures. (B) Illustration of our model. Neural activity in the periphery, x, is generated by passing the stimulus, s, through a set of neural tuning curves, f(s), and then adding zero-mean noise, ξ, which may be correlated between cells. This activity then propagates via feed-forward connectivity, described by the matrix W, to the next layer. The activity at the next layer, y, is generated by passing the inputs, W · x, through a nonlinearity g(⋅), and then adding zero-mean noise, η.

Fig 2. Not all population codes are equally robust against corruption by noise.

We constructed two model populations, each with the same 100 tuning curves for the first layer of cells but with different covariance structures, Σ_ξ (see text, especially Eq (4)). The covariance structures were chosen so that the two populations convey identical amounts of information I_x(s) about the stimulus. (A) 20 randomly-chosen tuning curves from the 100 cell population. (B) We corrupted the responses of each neural population by additional Gaussian noise (independently and identically distributed for all cells) of variance σ², to mimic corruption that might arise as the signals propagate through a multi-layered neural circuit, and computed the “output” information I_y(s) that these further-corrupted responses convey about the stimulus (blue and green curves). The population shown in green forms a relatively fragile code wherein modest amounts of noise strongly reduce the information, whereas the population shown in blue is more robust. (C) Input information I_x(s) in the two model populations (left; “correlated”) and information that would be conveyed by the model populations if they had their same tuning curves and levels of trial-to-trial variability, but no correlations between cells (right; “trial-shuffled”). For panels B and C, we computed the information for each of 100 equally spaced stimulus values, and averaged the information over those stimuli. See Methods for additional details (section titled “Details for Numerical Examples”).

Fig 3. Geometry of robust versus fragile population codes.

Cartoons showing the interaction of signal and noise for two populations with the same information in the input layer. The dimension of the space is equal to the number of cells in the population; we show a two dimensional projection. Within this space, when the stimulus changes by an amount Δs (with Δs small), the average neural response changes by f′(s)Δs. Thus, f′(s) is the “signal direction” (green arrows). Trial-by-trial fluctuations in the neural responses in the first layer are described by the ellipses; these correspond to 1 standard-deviation probability contours of the conditional response distributions. The impact of the neural variability on the encoding of stimulus s is determined by the projection of the response distributions onto the signal direction (magenta double-headed arrows). By construction, these are identical in the first layer. Accordingly, an observer of the neural activity in the first layer of either population would have the same level of uncertainty about the stimulus, and so both populations encode the same amount of stimulus information. When additional iid noise is added to the neural responses, the response distributions grow; the dashed ellipses show the resultant response distributions at the second layer. Even though the same amount of iid noise is added to both populations, the one in panel A shows greater stimulus uncertainty after the addition of noise than does the one in panel B. Consequently, the information encoded by the population in panel B is more robust against corruption by noise.

Fig 4. Family of optimal covariance matrices.

For all panels, green arrows indicate the signal direction, f′(s). Magenta ellipses indicate the noise in the first layer (with corresponding covariance matrix Σ_ξ), and grey ellipses indicate the effective noise in the second layer (with corresponding covariance matrix Σ_y). (A) The covariance ellipse in the first layer has its long axis aligned with the signal direction; this configuration (which corresponds to differential correlations) optimizes information robustness for any distribution of second layer noise. (B) The covariance ellipse in the first layer does not have its long axis aligned with the signal direction. However, the covariance ellipse of the effective noise in the second layer, Σ_y, has the same shape as the covariance ellipse in the first. In this case, the blue “good” projection—which is aligned both with a low-variance direction of the first-layer distribution (magenta), and with the signal curve (green), and thus is relatively informative about the stimulus (see text)—is corrupted by relatively little noise at the second layer. This “matched” noise configuration is among those that optimize robustness to noise. The optimal family of covariance matrices interpolates between the configurations shown in panels A and B. (C) Again the covariance ellipse in the first layer does not have its long axis aligned with the signal direction. But now the “good” projection is heavily corrupted by noise at the second layer. In this configuration, all projections are substantially corrupted by noise at some point in the circuit, and thus relatively little information can propagate.

Fig 5. Differential correlations enhance information propagation through “spike-generating” nonlinearities.

Responses in the second layer were generated using the dichotomized Gaussian model of spike generation, in which the input from the first layer was simply binarized via a step function (see Eq (11)). We varied the correlations in these inputs (see Eq (12)) while keeping the input information and input tuning curves fixed. (A) Heterogeneous tuning curves in the second layer, evaluated at ϵ_u = 0; we show a random subset of 20 cells out of the 100-neuron population studied in panel B. (B) Information transmitted by the 100-cell spiking population as a function of ϵ_u, which is the strength of the noise in the u(s) direction, for different angles, θ_u, between u and f′(s) (see Eq (12)). The input information was held fixed as ϵ_u was varied. The information is averaged over 20 evenly spaced stimuli (see Methods, section titled “Details for Numerical Examples”).

Fig 6. Information propagation through spike-generating nonlinearities with additive input noise.

As with Fig 5, responses in the second layer were generated using the dichotomized model of spike generation, in which the input from the first layer was simply binarized. Here, though, Gaussian noise was added before thresholding; see Eq (13). We varied the correlations in the input layer (see Eq (12)) while keeping the input information and input tuning curves fixed for the 100-cell population (same tuning curves and covariance matrices as in Fig 5). The additive noise at the second layer (the ζ_i) was iid Gaussian, with variance ${\sigma }_{\zeta }^2$; different colored lines correspond to different values of ${\sigma }_{\zeta }^2$. (A) Output information versus ϵ_u for populations with differential correlations (u = f′(s)). (B) Same as panel A, but for populations that concentrate noise along an axis, u, that makes an angle of 0.1 rad with the f′(s) direction. For both panels, the input information was held fixed as ϵ_u was varied, and the information was averaged over 20 evenly spaced stimuli.

Fig 7. Not all synergistic population codes are equally robust against corruption by noise.

This figure is similar to Fig 2, but with synergistic instead of redundant population codes. We constructed two model populations—each with the same 100 tuning curves (20 randomly-chosen example tuning curves are shown in panel A)—for the first layer of cells. The two populations have different covariance structures Σ_ξ for their trial-to-trial variability (see main text, Eq (4)), but convey identical amounts of information, I_x(s), about the stimulus. (B) We corrupted the responses of each neural population by Gaussian noise (independently and identically distributed for all cells) of variance σ², to mimic corruption that might arise as the signals propagate through a multi-layered neural circuit, and computed the output information, I_y(s), that these further-corrupted responses convey about the stimulus (blue and green curves). (C) Input information I_x(s) in the two model populations (left; “correlated”) and information that would be conveyed by the model populations if they had their same tuning curves and levels of trial-to-trial variability, but no correlations between cells (right; “trial-shuffled”). For panels B and C, we computed the information for 100 different stimulus values, equally spaced between 0 and 2π, and averaged the information over these stimuli.