Socially interactive industrial robots: a PAD model of flow for emotional co-regulation

Abstract

This article presents the development of a socially interactive industrial robot. An Avatar is used to embody a cobot for collaborative industrial assembly tasks. The embodied covatar (cobot plus its avatar) is introduced to support Flow experiences through co-regulation, interactive emotion regulation guidance. A real-time continuous emotional modeling method and an aligned transparent behavioral model, BASSF (Boredom, Anxiety, Self-efficacy, Self-compassion, Flow) is developed. The BASSF model anticipates and co-regulates counterproductive emotional experiences of operators working under stress with cobots on tedious industrial tasks. The targeted Flow experience is represented in the three-dimensional Pleasure, Arousal, and Dominance (PAD) space. We present how, despite their noisy nature, PAD signals can be used to drive the BASSF model with its theory-based interventions. The empirical results and analysis provides empirical support for the theoretically defined model, and clearly points to the need for data pre-filtering and per-user calibration. The proposed post-processing method helps quantify the parameters needed to control the frequency of intervention of the agent; still leaving the experimenter with a run-time adjustable global control of its sensitivity. A controlled empirical study (Study 1, N = 20), tested the model's main theoretical assumptions about Flow, Dominance, Self-Efficacy, and boredom, to legitimate its implementation in this context. Participants worked on a task for an hour, assembling pieces in collaboration with the covatar. After the task, participants completed questionnaires on Flow, their affective experience, and Self-Efficacy, and they were interviewed to understand their emotions and regulation during the task. The results from Study 1 suggest that the Dominance dimension plays a vital role in task-related settings as it predicts the participants' Self-Efficacy and Flow. However, the relationship between Flow, pleasure, and arousal requires further investigation. Qualitative interview analysis revealed that participants regulated negative emotions, like boredom, also without support, but some strategies could negatively impact wellbeing and productivity, which aligns with theory. Additional results from a first evaluation of the overall system (Study 2, N = 12) align with these findings and provide support for the use of socially interactive industrial robots to support wellbeing, job satisfaction, and involvement, while reducing unproductive emotional experiences and their regulation.

Keywords: affect modeling; affective computing; arousal; emotion (Co-)Regulation; human-robot interaction; pleasure; social signals; socially interactive agents.

FIGURE 1

Abstract view of the application setup.

FIGURE 2

Photos of the working space. The operator stands in front of his desk, where he joins his parts with the ones given by the cobot. The desk on his right is the operating space where the cobot picks his components. The tablet showing the avatar stands on the operator desk, just in front of the cobot rotating base. Left: phase 1 setup - The robot and the operator have completed their subassembly and the collaborative joining is ongoing. Right: phase 2 setup - The robot has brought the subassembly towards the operator, who is still working on his parts.

FIGURE 3

Avatar design and implementation. Left: a sketch for the design of the face. Center: an early off-line rendering of the character in Blender. Right: A screenshot of the real-time rendering in Unity.

FIGURE 4

Top: the top-level of the VSM project driving Cobot and Avatar behavior. Bottom: a view of the sub-node implementing the decision strategy of the BASSF model.

FIGURE 5

Boxplots comparing the calibration data among users.

FIGURE 6

Example of PAD streams over 30 min and their activations (vertical lines), for a single user, with $W=30$ seconds and $K=2$ .

FIGURE 7

Flow and PAD values over time for each participant, Study 1. The dotted vertical line represents the change from the slow to the fast phase.

FIGURE 8

Sum of the number of events over all users, grouped by event type.