Painterly depiction of material properties

Abstract

Painters are masters of depiction and have learned to evoke a clear perception of materials and material attributes in a natural, three-dimensional setting, with complex lighting conditions. Furthermore, painters are not constrained by reality, meaning that they could paint materials without exactly following the laws of nature, while still evoking the perception of materials. Paintings have to our knowledge not been studied on a big scale from a material perception perspective. In this article, we studied the perception of painted materials and their attributes by using human annotations to find instances of 15 materials, such as wood, stone, fabric, etc. Participants made perceptual judgments about 30 unique segments of these materials for 10 material attributes, such as glossiness, roughness, hardness, etc. We found that participants were able to perform this task well while being highly consistent. Participants, however, did not consistently agree with each other, and the measure of consistency depended on the material attribute being perceived. Additionally, we found that material perception appears to function independently of the medium of depiction-the results of our principal component analysis agreed well with findings in former studies for photographs and computer renderings.

Figure 1.

Examples of four stimuli. For the top two, the context size exceeds the dimensions of the original painting, and the overflow has been colored with the average RGB color value of the painting contained within the bounding box. For the bottom two, the context size does not exceed the original painting dimensions and is thus only a section of the painting without any overflow. The red outlines indicate the segments. From top-left to bottom-right: detail of David with the Head of Goliath (c. 1645) by Guido Cagnacci; The Explorer A.E. Nordenskiöld (1886) by Georg von Rosen; detail of Polyptych with Saint James Major, Madonna and Child, and Saints (1490) by Bartolomeo Vivarinil; and detail of Mlle Charlotte Berthier (1883) by Auguste Renoir.

Figure 2.

Distribution of completed rating tasks per participants.

Figure 3.

Example of the perceptual judgment task. At the top, the question and definition are repeated, which participants would have seen in the instructions. The task shows one segment at a time, as part of the original painting. In the live version, the red outline appears flashing at around 10 hz at the onset of each trial (or when the participant pressed the corresponding button) to indicate the segment boundaries and disappears after a second. The slider can be moved by moving the mouse up and down, whereas a left mouse-click progresses the experiment to the next trial. The painting is a section of The Annunciation (c.1660) by Godfried Schalcken.

Figure 4.

Each of the 50 set/attribute combinations expressed in a two-dimensional intraobserver/interobserver correlational space. The data are color-coded to indicate the material attribute that was judged. Ellipses (1 SD) are fitted for each material attribute based on the five experimental blocks relating to that attribute. The red lines represent the one-sided 5% alpha significance level, with 88° and 8° of freedom for intraobserver and interobserver correlations, respectively.

Figure 5.

Distribution of all the judgments per attribute for all materials. The colors are in reference to the colors used by Fleming et al. (2013).

Figure 6.

The averaged ratings for each attribute per material.

Figure 7.

A recreation of the average rating for the attributes and materials from Fleming et al (2013), for the materials and attributes that are shared between our study and Fleming's study. Note that in our study, we split up colorfulness into vivid and multicoloredness. Figure adapted with permission; original copyright belongs to ARVO.

Figure 8.

Correlation matrix heatmap, we have masked the values along the diagonal, which would always simply be 1 and the symmetrically identical values. * indicates p < 0.001, ** indicates p < 0.0001, and *** indicates p < 0.00001.

Figure 9.

A visualization of the first two PCA dimensions. The color of the points relates to material class identity. The factor loadings of the original dimensions are plotted as red vectors. Lastly, we fitted ellipsoids (sd = 1) for each material class. Note that the PCA is not fed any class data; the clustering of material classes observed is thus purely based on the perceptual data.

Figure 10.

Scree plot for the PCA visualized in Figure 9.

Figure 11.

Four visualizations of the first two primary components for the material-specific PCA for flora, fabric, paper, and skin. Each PCA was run with only the 30 stimuli per material. The red vectors indicate the factor loadings of each attribute. We plotted the actual stimuli within the PCA space. The blue lines connect the stimuli to their actual position within the space when the stimuli would otherwise overlap. The ellipse is fitted around the points (1 SD).

Figure 12.

Correlation between the perceived perceptual attributes and image statistics (i.e., image statistics). * indicates p < 0.0013, ** indicates p < 0.00013, and *** indicates p < 0.000013.