Statistics of natural scenes and cortical color processing

Abstract

We investigate the spatial correlations of orientation and color information in natural images. We find that the correlation of orientation information falls off rapidly with increasing distance, while color information is more highly correlated over longer distances. We show that orientation and color information are statistically independent in natural images and that the spatial correlation of jointly encoded orientation and color information decays faster than that of color alone. Our findings suggest that: (a) orientation and color information should be processed in separate channels and (b) the organization of cortical color and orientation selectivity at low spatial frequencies is a reflection of the cortical adaptation to the statistical structure of the visual world. These findings are in agreement with biological observations, as form and color are thought to be represented by different classes of neurons in the primary visual cortex, and the receptive fields of color-selective neurons are larger than those of orientation-selective neurons. The agreement between our findings and biological observations supports the ecological theory of perception.

Figure 1

The sizes of the filters used for extracting small-, medium-, and large-scale orientation and color fields.

Figure 2

(a) Spatial autocorrelation of the orientation field in natural images. The orientation is computed at three different spatial scales, ranging from large to small scale. The standard error for the measurements shown is too small to be meaningfully depicted in this figure. For instance, the standard error for the orientation correlation at small scale is 0.0028796 for a pixel distance of 1 and 0.00037322 for a pixel distance of 100. (b) Spatial autocorrelation of the color field in natural images. The color is computed at three different spatial scales, ranging from large to small scale. The original images were smoothed with a Gaussian filter of varying size, as described in Figure 1. The standard error is 0.0022356 for a pixel distance of 1 and 0.0039698 for a pixel distance of 100. (c) The spatial correlation statistics of orientation when the location of orientation vectors is randomized. (d) The spatial correlation statistics of color when the location of color vectors is randomized. In all these plots, the correlations for a given distance have been averaged over all directions. (e) The correlation statistics gathered over five Jackson Pollock paintings. (f) The correlation statistics computed over randomized versions of Jackson Pollock paintings. The locations of existing orientation and color vectors were randomized.

Figure 3

(a) Spatial autocorrelation of the orientation field in natural images, shown over specific directions. The large-scale filters were used for smoothing the image before orientation and color computations. (b) An enlarged version of the autocorrelation for orientation. Contour plots are used to depict these functions, using the MATLAB command contourf. (c) Spatial autocorrelation of the color field. (d) An enlarged view of the autocorrelation for color. (e) Spatial autocorrelation of luminance. (f) An enlarged view of the autocorrelation for luminance.

Figure 4

The correlation statistics for orientation and color computed over the images in the Berkeley Segmentation Dataset and Benchmark (Martin et al., 2001). Small-scale filter sizes were used in this computation, as shown in Figure 1. (a) The correlation statistics for orientation. (b) The correlation statistics for color.

Figure 5

The marginal probability density functions are displayed in the first two columns. The probability density function for orientation P(θ_e) is shown in the first column. The second column contains probability density functions for L, a, and b. The third column shows the pairwise product of the two marginal probability density functions in the first two columns. The fourth column shows the joint probability distribution for the variables in the first two columns. These plots show that the joint probability distributions appear similar to the marginal probability density functions, suggestive of statistical independence of the variables represented in the first two columns.

Figure 6

(a) The 2D correlation statistics of the joint orientation and color vector as described in Equation 5. (b) An enlarged view of the correlation function around the origin. (c) For this correlation plot, the locations of the 4D vectors were randomized. We show an enlarged view around the origin. (d) A 1D plot for the correlation, generated by summing the correlations within an annulus at a given radius of the function shown in (a).

Figure 7

The 2D correlation statistics of the joint orientation and color vector as described in Equation 5 are compared with the statistics of the individual color and orientation fields. A 1D plot for each case is generated by summing the correlations within an annulus at a given radius of the 2D correlation function.