Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex

Abstract

Neural responses in the visual cortex are variable, and there is now an abundance of data characterizing how the magnitude and structure of this variability depends on the stimulus. Current theories of cortical computation fail to account for these data; they either ignore variability altogether or only model its unstructured Poisson-like aspects. We develop a theory in which the cortex performs probabilistic inference such that population activity patterns represent statistical samples from the inferred probability distribution. Our main prediction is that perceptual uncertainty is directly encoded by the variability, rather than the average, of cortical responses. Through direct comparisons to previously published data as well as original data analyses, we show that a sampling-based probabilistic representation accounts for the structure of noise, signal, and spontaneous response variability and correlations in the primary visual cortex. These results suggest a novel role for neural variability in cortical dynamics and computations.

Keywords: Bayesian computations; V1; natural images; noise correlations; normative model; spontaneous activity; stochastic sampling; theory; variability; vision.

Figure 1

Schematic of the Model (A) The generative model describing the statistical structure of image patches (x). Images arise as a linear combination of Gabor-filter basis functions with intensities y$=\left\{y_1,\dots ,y_n\right\},$ whose contribution to the image is jointly scaled by a “contrast” variable, $z$, plus Gaussian white noise (see Experimental Procedures for details). (B) Probabilistic inference and the generation of membrane potentials and spike counts. The progression of four steps in the model is shown in the middle of the panel, advancing from the bottom toward the top. The activations of two example cells in red and purple (see the corresponding basis functions in A) are illustrated in two different trials using the same stimulus, $\mathbf{x}$ (left and right sides in B). Basis function activations, $\mathbf{y}$, are inferred by inverting the generative process shown in (A). Due to noise and ambiguity in the model, $\mathbf{y}$ cannot be inferred from the image with certainty; hence, the result of Bayesian inference is a posterior probability distribution, $\text{P}\left(\mathbf{y}|\mathbf{x}\right)$. Membrane-potential values, $\mathbf{u}$, represent stochastic samples from $\text{P}\left(\mathbf{y}|\mathbf{x}\right)$ through a weak non-linear transformation (inset), with independent samples drawn every ∼20 ms, corresponding to typical autocorrelation timescales of V1 neurons (Azouz and Gray, 1999) (For illustration, membrane potentials are plotted after smoothing with a 7-ms Gaussian kernel here. See also Experimental Procedures). Instantaneous firing rates, $\mathbf{r}$, are obtained from membrane potentials by a rectifying non-linearity (Carandini, 2004; inset). Spike counts are obtained by deterministically integrating firing rates across time over the duration of a trial: a spike is fired whenever the cumulative firing rate reaches an integer value (open circles on cumulative firing-rate traces and ticks in spike rasters, with the final spike counts shown at the right end of each raster). Note that while the distribution of neural responses (mean, variance, and covariance) remains unchanged across trials using the same stimulus, the actual time course of membrane potentials and the spike counts can vary substantially across trials due to stochastic sampling from the same underlying distribution. (C) Statistics of the joint activity of a pair of neurons. The two sides show the membrane-potential trajectories of the pair of neurons in the two trials presented in (B) plotted against each other, revealing the higher-order statistics of the joint distribution (e.g., non-zero correlations). Colored lines correspond to the membrane-potential trajectories shown in (B) (color shade indicates elapsed time), and dashed gray ellipses show the covariance underlying the stochastic trajectories (identical for the two trials). The center shows joint spike-count distribution of the same two cells across a large set of trials (circles) for the same stimulus. The two colored circles correspond to the spike counts obtained from the two trials shown at the two sides and presented in (B). Small jitter was added to integer spike counts for illustration purposes. Photo is from Istock.com/CliffParnell.

Figure 2

Key Features of Response Variability in Model Membrane Potentials (A) Three example images (identified by the color of their frame) are shown at increasing contrast levels from left to right. Increasing the contrast shifts the posterior over the inferred contrast level, $z$, away from zero (gray distribution curves, from light to dark). (B) Joint membrane-potential distributions of two example neurons (images at the end of axes show corresponding basis functions) for the three sample images in (A) at low contrast (colored diamonds, means; colored ellipses, covariances for the three images). Colors correspond to image frames in (A) compared to the prior distribution (black cross, mean; dashed black ellipse, covariance). The prior and the three posteriors strongly overlap; therefore, samples drawn from these distributions (gray circles, prior; red dots, posterior for image with red frame; gray dots, average posterior across 100 different images) and their means (crosses and diamonds) are indistinguishable. Inset on top shows the prior mean (black cross) and posterior means for the three natural image patches presented in (A) (colored diamonds). (C and D) Shown as in (B), but for two higher contrast levels. The posteriors for the three images increasingly deviate from the prior and each other: their mean moves further away from zero while their covariances (noise covariances) shrink and remain similar. Signal covariance (yellow dotted ellipse in D) is aligned with the covariance of the prior (black dashed ellipse). Individual posteriors tile the subspace covered by the spontaneous covariance, such that samples drawn from the average posterior (gray dots), but not those drawn from any individual posterior (red dots), still overlap with those from the prior (gray circles). Insets on top show prior mean (black cross) and posterior means for the three images in (A) (red, green, and blue diamonds) as well as for 100 other natural image patches (yellow diamonds). In contrast to the decrease in noise covariances, signal covariances (covariances of posterior means across stimuli) increase with increasing contrast levels.

Figure 3

Stimulus Onset Quenches Neural Variability (A and B) Periodic membrane-potential oscillations induced in an example neuron by a drifting sinusoid grating stimulus with preferred (A) and non-preferred (orthogonal to preferred) orientation (B) appearing after a blank image. Variability of responses is shown by their standard deviation (flanking gray area) for the model (top), and by individual trajectories in example trials (thin black lines) for the experimental data (bottom). Thick black (top) and gray (bottom) lines show across-trial average. Arrows mark stimulus onset. (C and D) Population analysis of the effect of stimulus onset on the variance of membrane potentials (C) and the Fano factor of spike counts (D). Arrows mark stimulus onset; thick black lines and flanking thin lines show population average and SE. (E) Direct comparison of spike-count Fano factors during spontaneous activity in response to a blank stimulus and evoked activity in response to high-contrast stimuli. Bars show population average, error bars indicate 95% bootstrap confidence intervals, ^∗p < 0.05. In each panel, the top plot shows the model results, and the bottom plot presents experimental data. Experimental data in (A)–(D) were reprinted by permission from Macmillan Publishers Ltd: Nature Neuroscience (Churchland et al., 2010, intracellular recordings in anesthetized cat). (E) presents an analysis of data from Ecker et al. (2010) (extracellular unit recordings in awake macaque). Fano factors in (D) and (E) were computed after mean matching (see Supplemental Experimental Procedures).

Figure 4

Stimulus Dependence of Neural Response Variability (A) Across-trial SD of peak response amplitudes of a population of cells (circles) for low-contrast gratings plotted against the SD for high-contrast gratings at the preferred (blue) and non-preferred (red) stimulus orientation. (B) Spike-count Fano factors (mean matched) for low- and high-contrast stimuli. (C) Dependence of membrane potential SD on grating orientation at high (solid black line) and low (solid gray line) contrast. For reference, membrane potential SD during spontaneous activity recorded in response to a blank stimulus is also shown (dashed gray line). (D and E) Mean and variance (black and blue lines in D) and Fano factor (E) of spike counts as a function of stimulus orientation relative to the preferred orientation of the cell. (B)–(E) show population averages (bars or lines), with error bars showing 95% bootstrap confidence intervals (B) and SE (C)–(E), ^∗p < 0.05. Experimental data in (A) and (C) were reproduced from Finn et al. (2007) with permission from Cell Press (intracellular recordings in anesthetized cat), and (B), (D), and (E) present analyses of data from Ecker et al. (2010) (extracellular unit recordings in awake macaque).

Figure 5

The Effect of Aperture on Response Reliability, Sparseness, and Signal Correlations (A) The response of a representative neuron to repeated presentation of an image sequence constrained to the classical receptive field (CRF, black) or combined non-classical receptive field (nCRF) and CRF stimulation (CRF + nCRF, red). Model plots from top to bottom show distribution of inferred contrast levels, $z$, across frames of the stimulus movie (histograms); the SD (shaded area) and mean of the membrane potential (dotted lines, error bars to the right show signal variability); and the trial-average firing rate (solid lines) of a representative neuron across time. Experimental data show trial-average firing rate. (B) Reliability of membrane-potential responses with CRF-only and combined nCRF + CRF stimulation. Inset (top) shows changes in the reliability for individual neurons. (C) Lifetime sparseness of firing rates with CRF-only and combined nCRF + CRF stimulation. Insets show changes in sparseness for individual neurons. (D) Distribution of separation angles between the mean response vectors of cell pairs with overlapping CRFs for CRF-only and combined nCRF + CRF stimulation. Arrows mark average separation angles. A higher separation angle means lower signal correlation. (B) and (C) show population averages (bars or lines) with error bars showing SE, ^∗p < 0.05. Experimental data in (A)–(C) were reproduced from Haider et al. (2010) with permission from Cell Press (intracellular recordings in anesthetized cat), and those in (D) were reprinted from Vinje and Gallant (2000) with permission from AAAS (extracellular recording from awake macaque).

Figure 6

Relationship between Signal, Noise, and Spontaneous Correlations (A) Dependence of correlations during spontaneous activity, r_spont, on spike-count signal correlations, r_sign. (B) Dependence of noise correlations during evoked activity, r_noise, on signal correlations. Bars show averages across cell pairs with signal correlations below or above the r_sign = 0.5 threshold, as shown on the x axis; error bars show SE, ^∗p < 0.05. Insets show the distribution of noise correlations; dashed line shows the mean of the distribution. Bottom panels present analyses of data from Ecker et al. (2010) (extracellular unit recordings in awake macaque).

Figure 7

Match between Spontaneous and Average Evoked Activity Multi-Unit Distributions Depends on Correlations and the Stimulus Ensemble Used (A) Kullback-Leibler (KL) divergence between aEA for natural image patches (aEA_natural) and SA (light gray bar), and between aEA_natural and a shuffled version of SA, preserving individual firing rates but destroying all correlations across electrodes (SA_shuffled, hatched bar). For reference, baseline KL divergence between two halves of SA data is also shown (dashed line). (B) KL divergence between aEA and SA under three different stimulus conditions: natural image patches (aEA_natural, light gray bar, same as in (A); random block noise images (aEA_noise, dark gray bar); and grating stimuli with various phases, orientations, and frequencies (aEA_grating, black bar). In all panels, bars show averages across animals and error bars show SE, ^∗p < 0.05. Bottom panels present analyses of experimental data from Berkes et al. (2011a) with permission from AAAS (extracellular multi-unit recordings in awake ferrets).