Socio-conversational systems: Three challenges at the crossroads of fields

Abstract

Socio-conversational systems are dialogue systems, including what are sometimes referred to as chatbots, vocal assistants, social robots, and embodied conversational agents, that are capable of interacting with humans in a way that treats both the specifically social nature of the interaction and the content of a task. The aim of this paper is twofold: 1) to uncover some places where the compartmentalized nature of research conducted around socio-conversational systems creates problems for the field as a whole, and 2) to propose a way to overcome this compartmentalization and thus strengthen the capabilities of socio-conversational systems by defining common challenges. Specifically, we examine research carried out by the signal processing, natural language processing and dialogue, machine/deep learning, social/affective computing and social sciences communities. We focus on three major challenges for the development of effective socio-conversational systems, and describe ways to tackle them.

Keywords: Affective computing; Machine learning; Multimodality; Natural language processing; Social signal processing; Socio-conversational systems.

FIGURE 1

The three challenges for gathering different research in conversational AI.

FIGURE 2

Two types of task-oriented systems architectures: modular (top) or end-to-end (below).

FIGURE 3

Machine learning approaches in socio-conversational systems. Orange boxes represent the different types of supervision of machine learning models, white boxes the different usages, green boxes the intervention of external knowledge, and blue and purple arrows represent the modular and end-to-end settings, respectively. Dotted arrows indicate when the information comes from labels derived from human knowledge.