The intrinsic variance of beauty judgment

Abstract

Recall memory and sequential dependence threaten the independence of successive beauty ratings. Such independence is usually assumed when using repeated measures to estimate the intrinsic variance of a rating. We call "intrinsic" the variance of all possible responses that the participant could give on a trial. Variance arises within and across participants. In attributing the measured variance to sources, the first step is to assess how much is intrinsic. In seven experiments, we measure how much of the variability across beauty ratings can be attributed to recall memory and sequential dependence. With a set size of one, memory is a problem and contributes half the measured variance. However, we showed that for both beauty and ellipticity, with set size of nine or more, recall memory causes a mere 10% increase in the variance of repeated ratings. Moreover, we showed that as long as the stimuli are diverse (i.e., represent different object categories), sequential dependence does not affect the variance of beauty ratings. Lastly, the variance of beauty ratings increases in proportion to the 0.15 power of stimulus set size. We show that the beauty rating of a stimulus in a diverse set is affected by the stimulus set size and not the value of other stimuli. Overall, we conclude that the variance of repeated ratings is a good way to estimate the intrinsic variance of a beauty rating of a stimulus in a diverse set.

Keywords: Aesthetics; Intrinsic variability; Recall memory; Repeated measures; Sequential dependence; Subjective beauty judgments.

Fig. 1

Graphical representation of the procedure of Experiment 1. Each box represents one block

Fig. 2

Histograms of the differences between the initial beauty rating and the beauty rating in the repeat block (A) and the memory block (B) and between the initial ellipticity rating and the ellipticity rating in the repeat (C) and memory (D) block

Fig. 3

Mean ellipticity ratings vs. aspect ratio, across stimuli in Experiment 2. Confidence intervals represent ± two standard errors

Fig. 4

Violin plots of the differences between the initial and scrambled-order (white) and same-order (gray) ratings for beauty of OASIS images (A; Experiment 3), ellipticity (B; Experiment 4), beauty of sunsets (C; Experiment 5), and beauty of photoshoot images (D; Experiment 6). The horizontal lines correspond to the medians and the vertical black rectangles correspond to the interquartile range

Fig. 5

Contrast and assimilation effect as a function of similarity. The x-axis represents Experiments 2, 5, 6, and 3, from left to right. As similarity between stimuli increases, the magnitude of both assimilation and contrast effects increases, regardless of task. * indicates p < 0.05. Similarity has a significant effect as indicated by the star, independently for each of the four image kinds. (Color figure online)

Fig. 6

Density distributions of the difference as a function of set size. The gray distributions correspond to the repeat blocks and the black distributions correspond to the memory blocks. The first column corresponds to a stimulus set size of 1 image, the middle column corresponds to a set size of 9 images, and the third column corresponds to a set size of 75 (Experiment 1, see Fig. 2)

Fig. 7

Standard deviations of the difference for the memory block, repeat block, and for the immediate-perception judgment as a function of stimulus set size, plotted on log-log scale. Error bars represent 95% confidence intervals of 500 bootstrap samples. The plot indicates an increase in repeated measures variance even after discounting the effect of memory