Affect-Driven Learning of Robot Behaviour for Collaborative Human-Robot Interactions

Abstract

Collaborative interactions require social robots to share the users' perspective on the interactions and adapt to the dynamics of their affective behaviour. Yet, current approaches for affective behaviour generation in robots focus on instantaneous perception to generate a one-to-one mapping between observed human expressions and static robot actions. In this paper, we propose a novel framework for affect-driven behaviour generation in social robots. The framework consists of (i) a hybrid neural model for evaluating facial expressions and speech of the users, forming intrinsic affective representations in the robot, (ii) an Affective Core, that employs self-organising neural models to embed behavioural traits like patience and emotional actuation that modulate the robot's affective appraisal, and (iii) a Reinforcement Learning model that uses the robot's appraisal to learn interaction behaviour. We investigate the effect of modelling different affective core dispositions on the affective appraisal and use this affective appraisal as the motivation to generate robot behaviours. For evaluation, we conduct a user study (n = 31) where the NICO robot acts as a proposer in the Ultimatum Game. The effect of the robot's affective core on its negotiation strategy is witnessed by participants, who rank a patient robot with high emotional actuation higher on persistence, while an impatient robot with low emotional actuation is rated higher on its generosity and altruistic behaviour.

Keywords: core affect; human-robot interaction; multi-modal affect perception; neural networks; reinforcement learning.

FIGURE 1

Proposed Framework: Multi-Modal Affect Perception of the robot combines facial and auditory features using the MCCNN. The Perception-GWR creates feature prototypes with the BMUs encoding arousal-valence while repeated interactions form the Affective Memory. The Affective Core models affective dispositions in the robot, resulting from its social conditioning and time perception. Current perception, affective memory and the affective core influence Mood formation which is used by the Behaviour Generation model to learn negotiating behaviour in the Ultimatum Game.

FIGURE 2

Patient (A) and Impatient (B) Mood Modulation results in Affective Cores encoding corresponding Patient (C) and Impatient (D) Time Perception for the agent.

FIGURE 3

Affective Cores encoding Excitatory or High-arousal (A) and Inhibitory or Low-arousal (B) modulation to form the Social Conditioning of the agent.

FIGURE 4

Participant and the NICO robot negotiating in the Ultimatum Game scenario.

FIGURE 5

Actor-Critic model for Learning Robot behaviour.

FIGURE 6

Arousal and Valence distributions for different Affective Core biases. All distributions are compared to the No Core condition to measure significant differences due to the Affective Core of the robot. A * denotes a significant variation compared to the No Core condition.

FIGURE 7

Robot learning to negotiate, converging an offer $>45\%$ of the resources in $\approx 10$ interactions.

FIGURE 8

Asch’s Test results with mean and 95% CI for individual dimensions comparing the two measured conditions. A * denotes a significant difference between the two conditions.