Hiding true emotions: micro-expressions in eyes retrospectively concealed by mouth movements

Abstract

When we encounter someone we dislike, we may momentarily display a reflexive disgust expression, only to follow-up with a forced smile and greeting. Our daily lives are replete with a mixture of true and fake expressions. Nevertheless, are these fake expressions really effective at hiding our true emotions? Here we show that brief emotional changes in the eyes (micro-expressions, thought to reflect true emotions) can be successfully concealed by follow-up mouth movements (e.g. a smile). In the same manner as backward masking, mouth movements of a face inhibited conscious detection of all types of micro-expressions in that face, even when viewers paid full attention to the eye region. This masking works only in a backward direction, however, because no disrupting effect was observed when the mouth change preceded the eye change. These results provide scientific evidence for everyday behaviours like smiling to dissemble, and further clarify a major reason for the difficulty we face in discriminating genuine from fake emotional expressions.

Figure 1. Experiment 1

(a) Four basic conditions. We independently manipulated changes in eye (upper) and mouth (lower) regions of a face across trials. In this figure, an eye region briefly changes into a happy expression in conditions 2 and 4 (micro-expression), while conditions 3 and 4 involve movements of a mouth. Participants judged presence/absence of the eye change ignoring the mouth change (eye detection task). (b) Relative timing of the eye and mouth changes in MC (mouth change) trials. The eye change started at 100 ms after onset of the mouth change. (c) Hit and false alarm (FA) rates in the eye detection task for six types of micro-expressions. The hit rates were decreased for all micro-expressions in the MC condition (red solid lines) compared to the NC condition (black solid lines), though FA rates were comparable between conditions (dotted lines). (d) The d’ obtained from the hit and FA rates. A two-way ANOVA of the six types of eye changes × mouth change (MC/NC) indicates a significant main effect of mouth change (MC < NC). There was no main effect of the eye changes or an interaction. In these and subsequent figures, all error bars denote standard errors (SEs) across participants. Note that, although faces in Figs 1, 2, 3, 4 are shown in line drawings for copyright reasons, all experiments were actually performed using photographs of real faces taken from a database developed for academic researches (ATR DB99). See our previous study for sample images from this database.

Figure 2. Experiment 2.

(a) Images presented to left and right eyes of a participant. Stereoscopic images when fused between two eyes are shown in the rightmost panels. In the behind condition (upper panels), upper and lower parts of a face were perceived to be stereoscopically behind a random-dot pattern, which enabled the participant to see a coherent image of a face behind an occluding pattern (amodal completion). This integrity (perceptual continuity) between the two parts was disconnected in the front condition (lower panels) where they were presented stereoscopically in front of the random-dot pattern. (b) The d’ in the eye detection task in the behind and front conditions. For all three types of micro-expression in eyes (anger, disgust, and happiness), a significant reduction in d’ by the mouth change was selectively observed in the behind condition. (c) The d’ combined across the three eye emotions. An ANOVA indicated a significant interaction between depth (behind/front) and mouth change (MC/NC). Post-hoc comparisons indicated a significant reduction of d’ in the behind but not in the front condition. **p < 0.01. ***p < 0.001.

Figure 3. Experiment 3 and Experiment 4.

(a) Experiment 3a (mouth change first). An inter-change interval from the offset of mouth change to the onset of eye change was randomly varied across trials from -200 to 0 ms. The d’ at four ICIs were not significantly different from the d’ in the control (NC) condition (dotted line). A background shading denotes SEs across participants in the NC condition. (b) Experiment 3b (eye change first). The detection of eye change was most severely impaired when the mouth change was delayed compared to the eye change by 50 ms. (c) Experiment 4. The d’ at each of six MC conditions and across the total of all MC conditions was decreased from that of the NC condition (dotted line). *p < 0.05. **p < 0.01. ***p < 0.001.

Figure 4. Perceptual bias induced by a smile.

Although the left and right faces share the same image of eyes with a neutral expression, the eyes on the right face are perceived to be happier than the left ones.