Interactions between faces and visual context in emotion perception: A meta-analysis

Abstract

Long-standing theories in emotion perception, such as basic emotion theory, argue that we primarily perceive others' emotions through facial expressions. However, compelling evidence shows that other visual contexts, such as body posture or scenes, significantly influence the emotions perceived from faces and vice versa. We used meta-analysis to synthesise and quantify these effects for the first time, testing if faces have primacy over context after accounting for key moderators. Namely, the emotional congruency and clarity of the stimuli. A total of 1,020 effect sizes from 37 articles and 3,198 participants were meta-analysed using three-level mixed-effects models with robust variance estimation. Both visual context and faces were found to have large effects on emotion labelling for the other (g_av > 1.23). Effects were larger when visual context and faces signalled different (incongruent) rather than the same (congruent) emotions and congruent effects were moderated by how clearly stimuli signalled the target emotion. When these factors were accounted for, faces were no more influential in altering emotion labelling than body postures or body postures with scenes. The findings of this review clearly evidence the integrative nature of emotion perception. Importantly, however, they also highlight that the influence of different emotion signals depends on how clearly they signal an emotion. Future research needs to account for emotional congruency and signal clarity.

Keywords: Affective integration; Facial primacy; Nonverbal communication.

Fig. 1

Calculation of the effect of context and the effect of faces. Photograph from RUN 4 FFWPU, care of Pexels

Fig. 2

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram

Fig. 3

Illustration of reverse scoring the incongruent effects. The histogram bars represent the frequency of the calculated effect sizes, binned by 0.25g_av. Each curve is the smoothed density estimate of the histograms. Effect sizes with Hedges g_av of -/ + 10 are not represented in this figure to optimise the visibility of most of the data

Fig. 4

Forest plot of incongruent (red) and congruent (blue) effect size estimates pooled by study. Arrows indicate that the 95% confidence interval falls outside of the included scale

Fig. 5

Interaction between congruency and clarity-difference scores. Clarity-difference represents the difference in proportion agreement between the added stimulus in isolation minus the agreement of the target stimulus in isolation. Positive clarity-difference scores reflect a clearer added stimulus, whereas negative clarity-difference scores reflect a clearer target stimulus. Zero reflects no difference in clarity between target and added stimulus