The geometry of masking in neural populations

Abstract

The normalization model provides an elegant account of contextual modulation in individual neurons of primary visual cortex. Understanding the implications of normalization at the population level is hindered by the heterogeneity of cortical neurons, which differ in the composition of their normalization pools and semi-saturation constants. Here we introduce a geometric approach to investigate contextual modulation in neural populations and study how the representation of stimulus orientation is transformed by the presence of a mask. We find that population responses can be embedded in a low-dimensional space and that an affine transform can account for the effects of masking. The geometric analysis further reveals a link between changes in discriminability and bias induced by the mask. We propose the geometric approach can yield new insights into the image processing computations taking place in early visual cortex at the population level while coping with the heterogeneity of single cell behavior.

Fig. 1

Measurement of population responses in masked and unmasked conditions. a Structure of the visual stimulus. Each of the lines show a single period of the stimulus in unmasked and masked conditions. b Examples of responses by individual neurons in both conditions. Periods of locomotion enhanced the overall responsivity of the population (shaded regions). Traces are plotted on a z-scored scale (vertical bar = 10). Horizontal bar represents 1 min of stimulation (or six periods of the orientation cycle). c Tuning in unmasked and masked conditions. Each trace shows the response of a neuron over the stimulation cycle after correction for neural delay, so they can be interpreted as a sweep of the orientation tuning curve of the neuron. The dashed line indicates the orientation of the mask. Blue traces represent the responses in the unmasked condition, while red traces represent responses in the masked condition. Shaded areas represent the mean response ± 2 SEM

Fig. 2

Characterization of responses in single neurons. a Anti-correlation between responses of neurons in masked and unmasked conditions. The mean responses of cells in the unmasked condition, μ_u, are anti-correlated with the responses in the masked condition, μ_m. The inset shows the distribution of ${\log }_2\left({\mu }_m∕{\mu }_u\right)$. Cells at the extreme of this distribution are termed grating (shaded green) and plaid (shaded pink) neurons. b Preferred orientation and average tuning of grating and plaid cells in unmasked and masked conditions. The histograms show the distribution of the preferred orientation of the neurons in each case. The red traces show the average tuning of neurons in each condition. The y-axis is labeled by cell count (in black) or by the amplitude of the responses (in red)

Fig. 3

Characterization of population responses. a Responses of a population of neurons in the unmasked and masked conditions. Cells were ordered according to their preferred orientation, thus resulting in a diagonal structure. The rows for these matrices represent the population responses in the unmasked and masked conditions, $r_u\left(\theta \right)$ and $r_m\left(\theta \right)$. These curves describe a close curve as $\theta$ describes one cycle. b. The intrinsic geometry of the curves is captured by the cosine distances between the representation of two orientations in the unmasked condition, $d_u\left(\theta ,\phi \right)$ (left panel), and masked condition, $d_m\left(\theta ,\phi \right)$ (middle panel). The relative positions of the curves with respect to each other is measured by the cosine distance between $r_u\left(\theta \right)$ and $r_m\left(\phi \right)$, denoted by $d_{um}\left(\theta ,\phi \right)$ (right panel). c. Simplified schematic showing the mean response to an orientation $r_u\left(\theta \right)$ along with its covariance matrix and the direction of the first and second eigenvectors, v₁ and v₂. d Results of two typical experiments showing the direction of the largest eigenvector (red dots) captured a substantial fraction of the of the variance (10–50%) and its direction is approximately aligned with the mean response (the cosine of the angle between the vectors was in the 0.8–1.0 range). Eigenvectors of higher rank (blue dots) accounted for significantly smaller fraction of the variance and their angles with respect to the mean response were much larger. e Population results. Average fraction of variance explained and the cosine of the angle with respect to the mean response averaged across all experiments. Error bars are about the size of the data points and not shown. f Simplified schematic in two-dimensions showing how the covariance of responses is anchored to the mean response of the vector population. The distribution of the cosine of the angle between v₁ and the direction of encoding $r^{\prime }\left(\theta \right)$ has a mode at zero, meaning these vectors tend to be orthogonal to each other

Fig. 4

Multidimensional scaling (MDS) of population responses in unmasked and masked conditions. Each row shows two viewpoints of the result of one experiment. The curves were obtained by performing MDS simultaneously on the population responses in unmasked and masked conditions into 3D space using the cosine distance as a metric. The blue curve shows $r_u\left(\theta \right)$ and the red curve shows $r_m\left(\theta \right)$. The gray sphere represents the origin, and colored spheres represent the beginning of the cycle. The green arrows represent the shift in the white point between conditions. The stimuli represent the patterns at different locations on the curves for the two conditions (blue outline—unmasked condition, red outline—masked condition). The icons on the bottom left represent the configuration of the stimulus along different parts of the curves

Fig. 5

Basic geometric properties of population representations in unmasked and masked conditions. a Shrinkage of the length of the curves by the introduction of the mask. Scatterplot shows the lengths of the curves in unmasked (L_u) and masked (L_m) conditions. Dashed line represents the unity line. Inset shows the distribution of L_m/L_u across all experiments. b Distribution of white-point shift ($\mathrm{\Delta }$) across all experiments. c Measurements of the angle between $r_m\left(\theta \right)$ and the plane $\mathrm{span}\left\{r_u\left(\theta \right),r_u\left(\pi ∕2\right)\right\}$ across all experiments. Solid line represents the mean, while the shaded area represents ± 2 SEM

Fig. 6

A geometric model of masking. a Discriminability (d-prime) between the representation of two orientations in unmasked (left panels) and masked (middle panels) conditions. The top panels show results for one experiment, while the ones at the bottom show the average across all our experiments. Iso-performance contour for the single experiment is shown at $d^{\prime }=4$. The iso-performance contours for the average behavior is shown at levels of $d^{\prime }=4,6,8$. The widening in the iso-performance contours in the masked condition reflect an increase in thresholds near the mask (which has an orientation of 90°). This is best shown in the panels on the right, which show the dependence of thresholds in masked (red) and unmasked (blue) conditions as a function of a base angle. In the average data the shaded areas represent ± 2 SEM. b Mutual distances and bias. Top panels show the mutual distance between orientations across masked and unmasked representations (d_um) and the expected bias from a decoder based on the distances. The non-diagonal structure of d_um is more evident in the average data (bottom left panel), showing the locations of the minima of the main diagonal (white, dashed line). Bottom right panel shows the average bias across all our experiments. Shaded areas represent 2 SEM. c Two-dimensional geometric model of population coding. The model assumes $r_u\left(\theta \right)$ and $r_m\left(\theta \right)$ are two circles in the plane. The displacement of their centers (white points) induce changes in the mutual distances inducing corresponding changes in threshold (middle panel) and bias (right panel). d The model can be extended by allowing displacement of the curves along a third dimension. e Two viewpoints of the same population activity in (d) but now normalized to yield ${\widehat{r}}_u\left(\theta \right)$ and ${\widehat{r}}_m\left(\theta \right)$

Fig. 7

A simple geometric transformation accounts for the effects of masking in neural populations. a Multidimensional scaling indicates the data can be faithfully embedded in five dimensions. The y-axis represents the correlation between mutual distances in the native space and the low-dimensional embedding. Solid curve represents mean across all experiments; shaded area represent ± 2 SEM. b Fits of an affine model to low-dimensional representations of ${\widehat{r}}_u\left(\theta \right)$ and ${\widehat{r}}_m\left(\theta \right)$ in four different experiments. In each case, ${\widehat{r}}_u\left(\theta \right)$ represents the population response in the unmasked condition (blue), ${\widehat{r}}_m\left(\theta \right)$ represents the population response in the masked condition (red), and ${\tilde{r}}_m\left(\theta \right)$ is the best fit to the response in the masked condition by means of an affine transform. The curves are 2D projections of the five-dimensional fits. (c) Distribution of the correlation between the mutual distances between points in ${\widehat{r}}_m\left(\theta \right)$ and corresponding points in the fit ${\tilde{r}}_m\left(\theta \right).$ The high correlation values agree with the visual impression in (b) that the fits are of excellent quality