Synergy, redundancy, and independence in population codes

Abstract

A key issue in understanding the neural code for an ensemble of neurons is the nature and strength of correlations between neurons and how these correlations are related to the stimulus. The issue is complicated by the fact that there is not a single notion of independence or lack of correlation. We distinguish three kinds: (1) activity independence; (2) conditional independence; and (3) information independence. Each notion is related to an information measure: the information between cells, the information between cells given the stimulus, and the synergy of cells about the stimulus, respectively. We show that these measures form an interrelated framework for evaluating contributions of signal and noise correlations to the joint information conveyed about the stimulus and that at least two of the three measures must be calculated to characterize a population code. This framework is compared with others recently proposed in the literature. In addition, we distinguish questions about how information is encoded by a population of neurons from how that information can be decoded. Although information theory is natural and powerful for questions of encoding, it is not sufficient for characterizing the process of decoding. Decoding fundamentally requires an error measure that quantifies the importance of the deviations of estimated stimuli from actual stimuli. Because there is no a priori choice of error measure, questions about decoding cannot be put on the same level of generality as for encoding.

Figure 1.

A diagram of neural encoding and decoding. A pair of neurons, 1 and 2, encode information about a stimulus, s(t), with spike trains, r₁(t) and r₂(t). This may be described by the conditional probability distribution of the responses given the stimulus p(r₁, r₂|s). Decoding is the process of trying to extract this information explicitly, which may be done by other neurons or by the experimentalist. This process is described by a function, F, that acts on r₁ and r₂ and gives an estimated version of the stimulus.

Figure 2.

Schematic of a scrambled decoding process. Six stimuli, {s}, are encoded by neural responses and mapped by a decoder onto six estimated stimuli, {s^est}. This mapping is one-to-one, so it preserves all the information in the stimulus. However, the estimates are scrambled, so that this decoder never gives the correct answer.

Figure 3.

Graphical presentations of synergy as a combination of other measures of independence. A, Following Equation 11, we can represent the synergy or redundancy of a pair of cells as a point in a plane with the axes and I(R₁; R₂). Because both of these measures are non-negative, only the top right quadrangle is allowed. Neurons that possess activity independence lie on points along the abscissa. Neurons that possess conditional independence lie on points along the ordinate. Information independence corresponds to the diagonal that separates the synergistic values from the redundant ones. B, Similarly, following Equation 16, we can also express the synergy as a point in a plane with the axes ΔI_noise and ΔI_signal. Because ΔI_signal is non-negative, only the top half plane is allowed.

Figure 5.

Examples of counter-intuitive values of D̂. For both examples, there are two stimuli and two neural responses. The probability of each stimulus is 1/2. A, One conditional joint response distribution, p(r₁, r₂|s), which results in the synergy of the cells being larger than D̂;I(R₁, R₂, S) = 1 bit; Syn(R₁, R₂) = 0.377 bits; D̂; = 0.161 bits. B, Another conditional joint response distribution, p(r₁, r₂|s), which results in D̂ being larger than zero when the noise correlations contribute net redundancy; I(R₁, R₂; S) = 0.311 bits; Syn(R₁, R₂) = -0.311 bits; D̂ = 0.053 bits.

Figure 4.

D̂ can be larger than the information that the cells encode about the stimulus. A, The conditional joint response distribution p(r₁, r₂|s), of two neurons responding to three stimuli. Each of the neurons responds with either zero, one, or two spikes. p(r₁, r₂) is the average of p(r₁, r₂|s) over the stimuli. The a priori probability of each of the stimuli equals 1/3. B, The conditional stimulus distribution for the cell pair, p(s|r₁, r₂), obtained using Bayes' rule. C, The conditional stimulus distribution that assumes no noise correlation, p_shuffle(s|r₁,r₂), obtained by inverting p(r₁|s)p(r₂|s) using Bayes' rule; see text for details. For this case, the information that both cells carry about the stimulus, I(R₁, R₂; S) equals 0.0140 bits, whereas D̂ equals 0.0145 bits.

Figure 6.

Cells may be synergistic but D̂ = 0. The conditional joint response distribution p(r₁, r₂|s) of two neurons responding to three stimuli, each with probability 1/3. In this case, the cells are synergistic but D̂ is zero: I(R₁, R₂; S) = 1.585 bits; Syn(R₁, R₂) = 0.415 bits; D̂ = 0 bits.