Biologically Inspired Emotional Expressions for Artificial Agents

Abstract

A special area of human-machine interaction, the expression of emotions gains importance with the continuous development of artificial agents such as social robots or interactive mobile applications. We developed a prototype version of an abstract emotion visualization agent to express five basic emotions and a neutral state. In contrast to well-known symbolic characters (e.g., smileys) these displays follow general biological and ethological rules. We conducted a multiple questionnaire study on the assessment of the displays with Hungarian and Japanese subjects. In most cases participants were successful in recognizing the displayed emotions. Fear and sadness were most easily confused with each other while both the Hungarian and Japanese participants recognized the anger display most correctly. We suggest that the implemented biological approach can be a viable complement to the emotion expressions of some artificial agents, for example mobile devices.

Keywords: artificial agent; artificial emotion expression; emotion recognition; ethological approach; ethorobotics; human-computer interaction; human-robot interaction.

Figure 1

The figure shows the dynamic changes of the agent during the emotion displays. The starting state is the same for all displays (happiness, fear, anger, surprise, sadness, neutral).

Figure 2

The figure shows the percentage of answers given to the open-ended questionnaire in terms of the types of answers. The red lines indicate the separation of scores 1–2, indicated by dark gray colors (words referring to not being-like agents) and scores 3–4, indicated with light gray (words referring to being-like agents). Answers receiving scores 3 and 4 were analyzed further. Score 4: naming an inner state or emotion (or the lack of emotion) explicitly; score 3: mentioning a term or phrase, which implicitly indicates some inner state but without naming a concrete emotion; score 2: indicating some contextual behavior that attributes a meaning to the video which cannot be directly observed; score 1: formal description of the observed behavior.

Figure 3

The categories of answers given for the Fear (A) and Anger (B) displays. The darker shades signify the correctness of the answers, the white column shows the answers that are neutral in the case of the emotion at hand, while the striped columns indicate somewhat opposing meanings.

Figure 4

The striped columns show the largest score given to the correct emotions (in case the same high scores were given to other emotions as well as to the correct one, the 1 point was divided by the number of emotions that got the same score), the black columns show the maximum 5 score given for the correct emotions, while the gray column depicts the percentage of answers in which the participants gave 5 scores for any emotions in case of the neutral display in the Hungarian group.

Figure 5

Analysis of sum of scores in the Hungarian group. The X sign shows the correct emotions in each display.

Figure 6

Confusion matrix of answers in the Hungarian group. The gray cells indicate the percentage at which each emotion received the biggest score. The correct emotions are indicated by dark borders.

Figure 7

The figure shows the scores given for the six displays, while the circles show the medians for the scores given to the correct emotions by the Hungarian participants. The * indicates outliers.

Figure 8

Ward's hierarchical cluster analysis, showing the similarities of scores given to each emotion by the Hungarian participants.

Figure 9

The striped columns show the largest score given to the correct emotions, the black columns show the maximum 5 score given for the correct emotions, while the gray column depicts the percentage of answers in which the participants gave 5 scores for any emotions in case of the neutral display in the Japanese group.

Figure 10

Analysis of sum of scores in the Japanese group. The X sign shows the correct emotions in each display.

Figure 11

Ward's hierarchical cluster analysis showing the similarities of scores given to each emotion by the Japanese participants.

Figure 12

Confusion matrix for the answers of the matched Japanese and Hungarian participants. The gray cells indicate the percentage at which each emotion received the biggest score. The correct emotions are indicated by dark borders.

Figure 13

The figure shows the scores given for the six displays, while the circles show the medians for the scores given to the correct emotions by the Japanese participants and the participants in the matched Hungarian group. The * indicates outliers.