Information theoretic approaches to understanding circuit function

Abstract

The analysis of stimulus/response patterns using information theoretic approaches requires the full probability distribution of stimuli and response. Recent progress in using information-based tools to understand circuit function has advanced understanding of neural coding at the single cell and population level. In advances over traditional reverse correlation approaches, the determination of receptive fields using information as a metric has allowed novel insights into stimulus representation and transformation. The application of maximum entropy methods to population codes has opened a rich exploration of the internal structure of these codes, revealing stimulus-driven functional connectivity. We speculate about the prospects and limitations of information as a general tool for dissecting neural circuits and relating their structure and function.

Figure 1

Information encoded by the neural response can be quantified by the mutual information between response and stimulus I(s, r). A. The response distribution is given by P(r) = ∫ P(s_i) P(r|s_i). When specifying a value of s, e.g. s_i, significantly reduces uncertainty, or narrows the distribution, about r, the mutual information between r and s is large (left). The more precisely that s specifies r, the larger the information (right). B. Information and correlations. Here, two stimuli s_A and s_B generate binary responses {r₁,r₂} with identical marginal distributions (p(r₁|s_A)=p(r₁|s_B) and p(r₂|sA)=p(r₂|s_B)), yet differing joint responses (p({r₁,r₂}|s_A) /= p({r₁,r₂}|s_B)). If correlations are ignored, I(s, r) = 0; if correlations are maintained and the stimuli are equally likely, then I(s, r) ∼ 0.23 bits. From left to right: neural response to s_A, with p(r₁=1|s_A)=p(r₂=1|s_A)=0.4 but response covariance p(r₁r₂=1|s_A)-p(r₁=1|s_A)p(r₂=1|s_A)=0.14; neural response to s_B, with p(r₁=1|s_B)=p(r₂=1|s_B)=0.4 but response covariance p(r₁r₂=1|s_B)-p(r₁=1|s_B)p(r₂=1|s_B)=-0.12; the neural response neglecting covariances is identical for each stimulus, carrying no information.

Figure 2. Finding stimulus dimensions with information maximization

A. The MID method searches for a direction in stimulus space, ie. a filter f. The stimulus s is high-dimensional and is described by some distribution, the grey cloud P(s). One then takes the component of s along a direction f. B. This projected stimulus, f ⊕ s, has a (one-dimensional) distribution P(f ⊕ s). This is known as the prior distribution. The spike-triggered stimuli (denoted in orange), have distribution P(f ⊕ s| spike). The goal is to determine f such that the Kullback-Leibler distance (see Box) between these two distributions is maximized; here, direction f ₁ improves the separation of the distributions compared with f₂. Finding the direction f that maximizes the Kullback-Leibler distance is equivalent to maximizing the mutual information between the projected stimulus and the occurrence of a spike. C. For a model cell with the filter shown on the left and a sigmoidal nonlinearity, driven with natural images, the MID method recovers the true filter considerably better than the spike-triggered average (with thanks to T. Sharpee).

Figure 3

A schematic of the dichotomous Gaussian model of population activity (simplified version). (A) N cells receive a common Gaussian input, as well as independent inputs; the summed input is compared to a threshold to either generate a spike or silence at each timestep. (B) Intriguingly, the output of this idealized model of the thresholding mechanism of spike generation produces highly different population statistics than the corresponding pairwise maximum entropy model [45,46]. The red and blue lines show the fraction out of N=50 model cells that spike simultaneously in a given timestep; both models are fixed to have the same firing rate and pairwise correlation [45].