Scene statistics and noise determine the relative arrangement of receptive field mosaics

Abstract

Many sensory systems utilize parallel ON and OFF pathways that signal stimulus increments and decrements, respectively. These pathways consist of ensembles or grids of ON and OFF detectors spanning sensory space. Yet, encoding by opponent pathways raises a question: How should grids of ON and OFF detectors be arranged to optimally encode natural stimuli? We investigated this question using a model of the retina guided by efficient coding theory. Specifically, we optimized spatial receptive fields and contrast response functions to encode natural images given noise and constrained firing rates. We find that the optimal arrangement of ON and OFF receptive fields exhibits a transition between aligned and antialigned grids. The preferred phase depends on detector noise and the statistical structure of the natural stimuli. These results reveal that noise and stimulus statistics produce qualitative shifts in neural coding strategies and provide theoretical predictions for the configuration of opponent pathways in the nervous system.

Keywords: computational model; efficient coding; mosaic; retina.

Fig. 1.

Spatial tiling of receptive fields is predicted by efficient coding. (A) Efficient coding model architecture. Images of natural scenes (plus input noise ${\sigma }_{\text{in}}$) are multiplied by linear filters ${\mathbf{w}}_j$, passed through a nonlinear function ${\eta }_j$, and perturbed by output noise ${\sigma }_{\text{out}}$, resulting in a firing rate $r_j$ for neuron $j$. (B) Examples of initial and optimized filters and nonlinearities, where the filters are initialized from a white noise distribution and converge to ON and OFF kernels with a center-surround pattern. The nonlinearities are initialized as unscaled (but slightly perturbed) softplus functions and converge to nonzero threshold values. (C) A plot of 100 kernels from a trained model with ${\sigma }_{\text{in}}$ = 0.4, ${\sigma }_{\text{out}}$ = 3.0. (D and E) Contour plot showing the tiling of the ON and OFF kernels. The contours of the two types of kernels are drawn where the normalized pixel intensity is $\pm$ 0.21. Orange circles and blue x’s indicate receptive field centers of mass for ON and OFF cells, respectively. Scale bar is width of one image pixel.

Fig. 2.

Spatial coordination of receptive field mosaics depends on input and output noise level. Receptive field centers for ON (orange circle) and OFF (blue x) cells under differing sets of noise parameters. (A) (${\sigma }_{\text{in}}$ = 0.02, ${\sigma }_{\text{out}}$ = 1.0), (B) (${\sigma }_{\text{in}}$ = 0.1, ${\sigma }_{\text{out}}$ = 1.0), (C) (${\sigma }_{\text{in}}$ = 0.4, ${\sigma }_{\text{out}}$ = 2.0), (D) (${\sigma }_{\text{in}}$ = 0.4, ${\sigma }_{\text{out}}$ = 3.0). The first two parameter sets result in aligned mosaics, while the latter two, at higher levels of output noise, are antialigned. Scale bar is width of one image pixel.

Fig. 3.

Optimal mosaic configurations transition from aligned to antialigned as a function of noise. (A) As output noise increases, optimal mosaic configurations shift from aligned to antialigned, and this holds over a range of input noise values. Color indicates median nearest neighbor distance, with darker indicating more clearly aligned (see Eq. 3 and Methods). The existence of a phase boundary between the two arrangements is clear. (B) Examples of optimal mosaic arrangements for representative input and output noise combinations. Symbols denote the corresponding locations in A. Note the existence of optimal configurations between aligned and fully antialigned (hearts) for some parameter values. Scale bar is width of one image pixel.

Fig. 4.

Optimal threshold increases with higher noise and heavier-tailed image distributions. (A and B) Contribution of a single neuron to the mutual information as a function of threshold. Stars mark optimal thresholds for multiple parameter values. (A) When the input distribution is Gaussian, the optimal threshold increases with higher output noise, even as overall information decreases. (B) When the input distribution is heavier-tailed, modeled with a Student’s t distributions with varying degrees of freedom, the optimal threshold again increases. (C) Schematic illustrating the effects of increased noise or outliers. With increased output noise, neurons’ SNR (black line, Upper Right) decreases, and efficient coding predicts that units should increase threshold and gain to reduce low SNR responses (peach). Similarly, when preactivations are heavy-tailed (Lower Right), efficient coding predicts that thresholds should increase and gains slightly decrease (SI Appendix, Fig. S11), since more mass is contained in outliers. Thus, both output noise and heavy tails lead to higher thresholds.

Fig. 5.

Pairwise mutual information corrections explain the phase transition between alignment and antialignment. (A) Normalized pairwise information correction ($-N{h}_{2^{\prime }}/p_1^2$) for pairs of ON and OFF neurons as a function of mosaic shift. One unit is the spacing between nearest-neighbor receptive field centers of the same polarity. With low output noise, an aligned position (i.e., no shift) is the only placement to avoid a large negative information correction, while with higher output noise an ON–OFF pair can still have a finite shift within the gray area without incurring any penalty. (B) The total normalized pairwise information correction as a function of mosaic shift, assuming equispaced ON and OFF receptive field centers filling the one-dimensional space. With low output noise, any mosaic shift lowers mutual information, while with high output noise the pairwise mutual information becomes less negative when the mosaic is shifted by 0.5, i.e., when antialigned. (C) The optimal mosaic shift as a function of output noise. For ${\sigma }_{\text{out}}=1.7,{\sigma }_{\text{out}}=2$, the optimal mosaic shift lies between 0 (alignment) and 0.5 (antialignment), resembling the arrangements we obtained in the middle row of Fig. 3B. (D) Schematic of optimal two-dimensional mosaics in a low-noise regime. The gray circle indicates the area within which an OFF cell can be located with zero pairwise correction to mutual information. (E) Schematic of optimal two-dimensional mosaics in a high-noise regime. Here, the gray area has grown (as in A), allowing three neighbors of the opposite polarity to fit inside without loss in mutual information.

Fig. 6.

Outlier image patches drive an antialigned relationship between ON/OFF receptive field mosaics via higher optimal thresholds. (A) Distribution of all 18,587,848 image patches from the dataset, plotted as a two-dimensional histogram with respect to the mean and the SD of each image patch (orange). A few example patches are shown in the lower left, with the corresponding locations in the histogram marked with matching symbols. The means are centered at zero (SD = 0.575), and the SD values are centered at 0.623 (SD = 0.187). Ellipses in the histogram represent data within one to three SDs of the mean, which we use to construct artificial datasets with varying contributions of outliers. (B) Optimized ON and OFF receptive field mosaics using the image patches within the boundaries denoted in A with ${\sigma }_{\text{in}}$ = 0.4, ${\sigma }_{\text{out}}$ = 1.4 to 2.0, plotted with the same scheme as in Fig. 3. As predicted, when trained on image sets that systematically excluded outliers (i.e., had lighter-tailed distributions), phase transitions happened only at larger values of output noise. The fourth column denoted as “all patches” used the full dataset as in Fig. 3. The last column, denoted as “boosted outliers,” used an augmented dataset to which we added horizontal and vertical mirror flips of patches outside the 1.5 SD circle, thereby creating a distribution with even more outliers. This corresponds to the top 20% of z-scores among all patches in the dataset. Here, the phase transition happened at lower output noise values. The scale bar shows a one-pixel distance. (C) Optimal thresholds as a function of output noise, using input distributions with different levels of outliers. As predicted, the optimal thresholds tended to increase as the input distribution contained more outliers, causing the phase transition to happen at a lower output noise level.