Facial representations of complex affective states combining pain and a negative emotion

Abstract

Pain is rarely communicated alone, as it is often accompanied by emotions such as anger or sadness. Communicating these affective states involves shared representations. However, how an individual conceptually represents these combined states must first be tested. The objective of this study was to measure the interaction between pain and negative emotions on two types of facial representations of these states, namely visual (i.e., interactive virtual agents; VAs) and sensorimotor (i.e., one's production of facial configurations). Twenty-eight participants (15 women) read short written scenarios involving only pain or a combined experience of pain and a negative emotion (anger, disgust, fear, or sadness). They produced facial configurations representing these experiences on the faces of the VAs and on their face (own production or imitation of VAs). The results suggest that affective states related to a direct threat to the body (i.e., anger, disgust, and pain) share a similar facial representation, while those that present no immediate danger (i.e., fear and sadness) differ. Although visual and sensorimotor representations of these states provide congruent affective information, they are differently influenced by factors associated with the communication cycle. These findings contribute to our understanding of pain communication in different affective contexts.

Figure 1

Stimuli and measures of the three computer tasks: Virtual Agents, Posed Face, and Imitated Face. Participants were asked to represent the facial configuration most likely expressed by characters in the scenarios on the virtual agents' faces (Virtual Agents task) and on their own faces (Posed Face task). They also had to imitate the facial configurations previously created on the virtual agents (Imitated Face task). The facial movements (i.e., AUs) were extracted from the Expressive EEVEE application (Virtual Agents task) and photographs of the participants’ faces (Posed Face and Imitated Face tasks).

Figure 2

Absolute mean of SHAP indicating the relative importance of the intensity of the 12 AUs (or clusters of AUs) depicted on the virtual agents to predict affective states.

Figure 3

Results of the Affective states effect on the mean intensity of AUs depicted on the virtual agents. Colored dotted lines in the radar plot indicate a 95% CI for each affective state. The AUs with significant differences between affective states are shown beside the radar plot. The colored box-and-whisker plots and the colored points show the data distribution for each affective state (n = 28). The whiskers present the minimum and maximum values, the vertical length of the box presents the interquartile range, and the horizontal line within the box presents the median. The grey squares show the mean scores for each affective state and error bars indicate a 95% CI (Bootstrap = 1000) calculated by Seaborn. * ps_bonf < 0.05.

Figure 4

Absolute mean of SHAP indicating the relative importance of the intensity of the 12 AUs (or clusters of AUs) measured on the participants’ faces to predict affective states.

Figure 5

Absolute mean of SHAP indicating the relative importance of the intensity of the 12 AUs (or clusters of AUs) measured on the participants’ faces to predict the type of facial configurations.

Figure 6

Results of the Affective states effect and the interaction effect with Type of facial configurations on the mean intensity of AUs measured on participants’ faces. Colored dotted lines in the radar plot indicate a 95% CI for each affective state. The AUs with significant differences between affective states are shown beside the radar plot. The colored box-and-whisker plots and the colored points show the data distribution for each affective state on imitated (n = 27) and posed (n = 28) facial configurations. The whiskers present the minimum and maximum values, the vertical length of the box presents the interquartile range, and the horizontal line within the box presents the median. The grey squares show the mean scores for each affective state on imitated and posed facial configurations, and error bars indicate a 95% CI (Bootstrap = 1000) calculated by Seaborn. *ps_bonf < 0.05.

Figure 7

Results of the interaction effect of Affective states and Type of facial representation on the mean intensity of pain index. The colored box-and-whisker plots and the colored points show the data distribution for each affective state on the virtual agents and the participants’ faces (n = 28). The whiskers present the minimum and maximum values, the vertical length of the box presents the interquartile range, and the horizontal line within the box presents the median. The grey squares show the mean scores for each affective state on the virtual agents and the participants’ faces, and error bars indicate a 95% CI (Bootstrap = 1000) calculated by Seaborn. * ps_bonf < 0.05.