What is beautiful is still good: the attractiveness halo effect in the era of beauty filters

Abstract

The impact of cognitive biases on decision-making in the digital world remains under-explored despite its well-documented effects in physical contexts. This paper addresses this gap by investigating the attractiveness halo effect using AI-based beauty filters. We conduct a large-scale online user study involving 2748 participants who rated facial images from a diverse set of 462 distinct individuals in two conditions: original and attractive after applying a beauty filter. Our study reveals that the same individuals receive statistically significantly higher ratings of attractiveness and other traits, such as intelligence and trustworthiness, in the attractive condition. We also study the impact of age, gender and ethnicity and identify a weakening of the halo effect in the beautified condition, resolving conflicting findings from the literature and suggesting that filters could mitigate this cognitive bias. Finally, our findings raise ethical concerns regarding the use of beauty filters.

Keywords: artificial intelligence; attractiveness halo effect; beauty filters; cognitive biases; gender stereotypes.

Figure 1.

Overview of the study and the analysis of the collected data. The stimuli consist of two sets of facial images: the PRI dataset, extracted from existing datasets for research on faces [53,54] and the POST set, created by applying a state-of-the-art beauty filter to each image in the PRI dataset. Each participant (N = 2748) rated 10 different images on seven attributes indicated on the top right part of the figure. Each image received ratings from at least 25 different participants. To shed light on the attractiveness halo effect, two levels of analysis were performed: (i) an aggregate level—depicted inside the pink box in the figure—using the medians of all the ratings received by each image, which are referred to as centralized ratings (); and (ii) an individual level ()—depicted inside the blue box in the figure—consisting of each rating and considering the participants’ characteristics.

Figure 2.

Impact of the beauty filters on perceived attractiveness. The size of the circles is proportional to the number of ratings provided for each value on the 7-point Likert scale and the colour indicates the proportion of males and females for each rating. (a) Pairwise comparison of perceived attractiveness before and after beautification. Observe how no image decreased its perceived attractiveness ratings after beautification and how the highest perceived attractiveness ratings tend to correspond to females. (b) Increase in perceived attractiveness (${\mathrm{\Delta }}_{attrac}$) after the application of the beauty filter versus the initial levels of attractiveness. Shading corresponds to the $95\%$ confidence interval. The higher the original perceived attractiveness, the lower the increase in attractiveness after applying the filter.

Figure 3.

Distribution of the median ratings of perceived attractiveness of the original (PRI, in orange) and beautified (POST, in purple) face images when varying the age (a), gender (b) and ethnicity (c) of the stimuli. Note that the age and ethnicity results are computed on the FACES and CFD datasets, respectively, whereas the gender results are based on the analysis of both datasets. Regarding age, the younger the individual, the higher their perceived attractiveness ratings ($p<0.001$, pairwise Wilcoxon). With respect to gender, female faces receive higher attractiveness ratings than male faces ($p<0.001$, Kruskal–Wallis). No statistically significant difference was found in the attractiveness levels depending on the ethnicity of the stimuli both before and after beautification.

Figure 4.

Impact of rater’s and stimulus’s gender on attractiveness and the dependent variables in the PRI and POST datasets. The x-axis represents the gender of the rater and the colours represent the gender of the stimulus (pink [] for images of females and blue [] for images of males). The length of the bars corresponds to the 95% confidence interval of the estimated marginal mean (EMM) [70,71]. The y-axis depicts the relative change in the EMM from the EMM of female stimuli rated by female participants. Details on how these values were computed can be found in appendix N.

Figure 5.

A visual representation of the relationship between perceived attractiveness and the dependent attributes after rescaling with the ordered stereotype model. The scales here have been normalized for ease of representation. Note how intelligence shows a much stronger saturation effect than the other dependent attributes in the PRI dataset. In the POST dataset, both intelligence and trustworthiness exhibit a saturation effect.

Figure 6.

Samples of male (top) and female (bottom) face images used in our study before (left) and after (right) the application of the beauty filter. As illustrated in the examples, the beauty filter modifies the skin tone, the eyes and eyelashes, the nose, the chin, the cheekbones and the lips in order to make the person appear more attractive.

Figure 7.

Distribution of attractiveness ratings of (a) the self-rated attractiveness of the participants and (b) the attractiveness ratings provided to images in the PRI set.

Figure 8.

The new scales for perceived attractiveness and the dependent variables after rescaling by means of OSMs based on the collected data. The black dots at the bottom indicate the original, equally spaced 7-point Likert scale. The orange and purple dots correspond to the new scale for the PRI () and POST () sets, respectively. The squares around the dots indicate locations where multiple points on the scale were collapsed to the same value.

Figure 9.

Comparison of the perceptions of (a) femininity and (b) unusualness before (x-axis) and after (y-axis) the application of beauty filters.

Figure 10.

A summary of the characteristics of the 2748 participants of our study.

Figure 11.

A screen shot of the survey tool. The face image on the left remains static on the screen as participants scroll through the column on the right to answer all the questions described in §4.4 about each image. A progress bar on the top indicates progress in answering the questions.

Figure 12.

Projections of all the dependent attributes on the first two dimensions.

Figure 13.

Visualization of the pairwise change in centralized scores of the dependent variables after applying the beauty filters. The x-axis represents the score an image received in the PRI dataset and the y-axis represents the score the corresponding image received in the POST dataset. The size of the circles is proportional to the number of images with the PRI and POST scores represented by the point and the colour of the circles represents the proportion of males and females at that point.

Figure 14.

Three-level taxonomy of modelling choices that were evaluated for their goodness of fit, resulting in 10 different models.

Figure 15.

Relationship between attractiveness and the dependent attributes after rescaling the data on the PRI set. The yellow curve represents the rescaled data, the blue curve represents a logarithmic curve fit to the data and the dashed lines represent the best fit lines on the lower (black) and upper (red) half of the data. While all attributes show saturation to a degree, it is strongest for intelligence and trustworthiness. Note that the y-axis values for the different dependent attributes are not directly comparable since they were all rescaled independently using the OSMs.