Low level visual features support robust material perception in the judgement of metallicity

Abstract

The human visual system is able to rapidly and accurately infer the material properties of objects and surfaces in the world. Yet an inverse optics approach-estimating the bi-directional reflectance distribution function of a surface, given its geometry and environment, and relating this to the optical properties of materials-is both intractable and computationally unaffordable. Rather, previous studies have found that the visual system may exploit low-level spatio-chromatic statistics as heuristics for material judgment. Here, we present results from psychophysics and modeling that supports the use of image statistics heuristics in the judgement of metallicity-the quality of appearance that suggests an object is made from metal. Using computer graphics, we generated stimuli that varied along two physical dimensions: the smoothness of a metal object, and the evenness of its transparent coating. This allowed for the exploration of low-level image statistics, whilst ensuring that each stimulus was a naturalistic, physically plausible image. A conjoint-measurement task decoupled the contributions of these dimensions to the perception of metallicity. Low-level image features, as represented in the activations of oriented linear filters at different spatial scales, were found to correlate with the dimensions of the stimulus space, and decision-making models using these activations replicated observer performance in perceiving differences in metal smoothness and coating bumpiness, and judging metallicity. Importantly, the performance of these models did not deteriorate when objects were rotated within their simulated scene, with corresponding changes in image properties. We therefore conclude that low-level image features may provide reliable cues for the robust perception of metallicity.

Figure 1

Conjoint measurement analysis of the judgment of metallicity by observers. (a) The parameter space used in experiments. Stimuli were physically-based rendered computer graphics of a composite coated metal object. Smoothness of the metal, and bumpiness of the coating, vary from zero to a physically plausible upper limit. The objects could be viewed from eight possible rotations around the z-axis. (b) Conjoint measurement plots for individual observers. Plots show the contributions of metal smoothness (green) and coating bumpiness (blue) to judgments of metallicity. Error bars for individual participants show 95% confidence intervals obtained via bootstrap. The lower right-hand plot shows the mean of normalized estimates for P1–4, with error bars showing ± s.e.m. ($N=4$).

Figure 2

Spectral analysis of the stimuli used for conjoint measurement. Power spectra are shown for varying levels of (a) metal smoothness (green scale bar) and (b) coating bumpiness (blue scale bar) of the stimuli, interpolated onto a one-dimensional scale of radial frequency. Left: Spectra have been averaged across all eight viewing angles of the objects. Right: The power spectra for each individual viewing angle are shown separately, over the region shaded in grey in the left-hand plots. The mean power over the range shown is given by a horizontal line.

Figure 3

Steerable pyramid synthesis, starting from an image at the center of the stimulus space ($P_0^0$). Steerable pyramid analysis of the stimuli used for conjoint measurement. (a) Steerable pyramid synthesis, starting from an image at the center of the stimulus space ($P_0^0$). The synthesis process attempts to match the pyramid parameters of that image with the parameters of another pyramid, in this case from four different sides of the stimulus space, reaching smooth metal ($P_s^0$), rough metal ($P_r^0$), even coatings ($P_0^e$), bumpy coatings ($P_0^b$). (b) Heatmaps showing filter responses summed over orientations, for each level of the steerable pyramid. Values of each level have been normalized to the lowest physical scale value, to show the relative changes throughout the stimulus space, averaged across all viewing angles of the objects. (c) The response of Level 1 filters is a good predictor for metal smoothness, while the comparison of Level 1 and 4 filters is a good predictor for coating bumpiness, across all viewing angles. For each plot, individual data series are for a single viewing angle.

Figure 4

Comparison of computational models and observers for difference scaling of the dimensions of the stimulus space. (a) Simulated difference scaling using models of metal smoothness and coating bumpiness. The models use only the output of steerable pyramid levels for comparing trials. (b) Difference scaling results obtained from 5 observers completing an MLDS task. Error bars show 95% confidence intervals obtained via bootstrap.

Figure 5

Modeling observer judgments of metallicity. (a) The computational model of metallicity, taking a weighted comparison of the outputs of Level 1 and Level 4 of the steerable pyramid decomposition. The pyramid is shown on the left, with higher levels resulting from applying a Gaussian blur to preceding levels and downsampling the spatial resolution by a factor of 2 (not to scale). The same linear oriented filters operate on all levels, such that lower levels generate responses owing to an image’s fine details, and higher levels generate responses owing to coarse features. The model has only two free parameters, a and b, the weightings of responses for Level 4 (coarse features) and Level 1 (fine details). (b) This model accounts for the observer judgments of metallicity. Conjoint measurement contributions are shown for metal smoothness (green) and coating bumpiness (blue). Observer estimates obtained via MLCM are shown in faded colors as in Fig. 1, with computational model fits shown in dotted lines. Also shown are the conjoint measurement contributions for computational models using only a single level of the pyramid. Level 1, corresponding to fine image features, at the top right, and Level 4, corresponding to coarse image features, at the bottom right. Error bars for pooled participant data show ± s.e.m. ($N=4$) and for individual observers show 95% confidence intervals obtained via bootstrap.

Figure 6

Comparing the effect of roughness on glass and silver objects. Glass (top row) and silver (bottom row) spheres have been rendered with varying physical roughness, increasing to the right. The objects have a similar luminance profile when completely smooth, making it hard to discern which material is which. However, increasing roughness has a more pronounced effect on the glass object than the silver object, as the luminance distribution is more tightly constrained, despite a similar apparent level of image blurring.