Socially intelligent robots: dimensions of human-robot interaction

Abstract

Social intelligence in robots has a quite recent history in artificial intelligence and robotics. However, it has become increasingly apparent that social and interactive skills are necessary requirements in many application areas and contexts where robots need to interact and collaborate with other robots or humans. Research on human-robot interaction (HRI) poses many challenges regarding the nature of interactivity and 'social behaviour' in robot and humans. The first part of this paper addresses dimensions of HRI, discussing requirements on social skills for robots and introducing the conceptual space of HRI studies. In order to illustrate these concepts, two examples of HRI research are presented. First, research is surveyed which investigates the development of a cognitive robot companion. The aim of this work is to develop social rules for robot behaviour (a 'robotiquette') that is comfortable and acceptable to humans. Second, robots are discussed as possible educational or therapeutic toys for children with autism. The concept of interactive emergence in human-child interactions is highlighted. Different types of play among children are discussed in the light of their potential investigation in human-robot experiments. The paper concludes by examining different paradigms regarding 'social relationships' of robots and people interacting with them.

Figure 1

People have long been interested in machines that simulate natural processes, in particular machines that simulate human behaviour and/or appearance. (a, b) The famous trumpet player designed by Friedrich Kaufmann in Dresden, Germany, (source: http://www.deutsches-museum.de/ausstell/meister/e_tromp.htm; copyright Deutches Museum, Munich). A variety of other androids were created trying to simulate appearance and behaviour of humans (and other animals), based on clockwork technology available at the time. Pierre Jaquet-Droz and Jacques de Vaucanson are among the famous designers of early androids in the eighteenth century. (c) A recent example, using the latest twenty-first century robotics technology, to simulate aspects of human behaviour: the Toyota robot at the Toyota Kaikan in Toyota City (This Wikipedia and Wikimedia commons image is from the user Chris 73 and is freely available at http://commons.wikimedia.org/wiki/Image:Toyota_Robot_at_Toyota_Kaikan.jpg under the creative commons cc-bu-sa 2.5 license.)

Figure 2

Experimental platforms that have been used widely in ‘nouvelle AI’ research: (a) Khepera and (b) Koala, both from K-Team (http://www.k-team.com). Both robots have two degrees of freedom that allow wandering in the environment. Optionally, grippers can be fitted to pick up objects.

Figure 3

Experimental humanoid robot platform for the study of synchronization, turn-taking and interaction games inspired by child development. Kaspar, a child-sized humanoid robot developed by the Adaptive Systems Research Group at the University of Hertfordshire. (a) Kaspar has a minimally expressive head with eight degrees of freedom in the neck, eyes, eyelids and mouth. The face is a silicon rubber mask, which is supported on an aluminium frame. It has two degrees of freedom in the eyes fitted with video cameras and a mouth capable of opening and smiling. It has six degrees of freedom in the arms and hand and is thus able to show a variety of different expressions. (b) Kaspar's expressions: happy; neutral; and surprised (Blow et al. 2006). (c) Some of Kaspar's expressions using movements in the head and arms.

Figure 4

Increasing requirements for social skills in different robot application domains.

Figure 5

Evaluation criteria to identify requirements on social skills for robots in different application domains. Contact with humans ranges from none, remote contact (e.g. for robots operating in deep-sea environments) to long-term, repeated contact potentially involving physical contact, as is the case, for example, in assistive robotics. The functionality of robots ranges from limited, clearly defined functionalities (e.g. as vacuum cleaning robots) to open, adaptive functions that might require robot learning skills (e.g. applications such as robot partners, companions or assistants). Depending on the application, domain requirements for social skills vary from not required (e.g. robots designed to operate in areas spatially or temporally separated from humans, e.g. on Mars or patrolling warehouses at night) to possibly desirable (even vacuum cleaning robots need interfaces for human operation) to essential for performance/acceptance (service or assistive robotics applications).

Figure 6

The conceptual space of HRI approaches. A, socially evocative; B, socially situated; C, sociable; D, socially intelligent; E, socially interactive (see text for explanations). Note: any robotic approach that can possibly be located in this framework also involves a more or less strong robotics component, i.e. the robot needs to be able to perform behaviours and tasks which can involve substantial challenges in the cases of a robot that possesses a variety of skills, as required, for example, for robots in service applications. This is less so in the cases where, for example, HRI research can be carried out with simple toy-like robots, such as Lego robots.

Figure 7

Challenges for a robot companion at the intersection of human-centred and robot cognition-centred views. The right balance needs to be found between how the robot performs its tasks as far as they are perceived by humans (human point of view) and its cognitive abilities that will determine, e.g. decision-making and learning (robot cognition view).

Figure 8

(a) Negotiated space task and (b) assistance task.

Figure 9

Layout of the experimental room for the negotiated space and assistance tasks. The room was provided with a whiteboard (9) and two tables. One table was furnished with a number of domestic items—coffee cups, tray, water bottle, kettle, etc. The other table (2) was placed by the window to act as a desk for the subject to work at while performing the assistance task, a vase with flowers, a desk light, and a bottle and glass of water were placed on the table. The room also included a relaxing area, with a sofa (3), a small chair and a low rectangular coffee table. Directly opposite, next to the whiteboard, was another low round coffee table, with a television placed on it. A second small chair stood in the corner. Five network video cameras were mounted on the walls in the positions indicated, recording multiple views of the experiment.

Figure 10

The Labo-1 robot used in the trials on playful interaction games with children with autism. The robot is 38×28 cm large, 21 cm high, mass 6.5 kg and has four wheels. Its four-wheel differential drive allows smooth turning. The robot has eight active infrared sensors positioned at the front (four sensors), rear (two) and one sensor on each side. A pyroelectric heat sensor was mounted on the front end of the robot and enabled it to detect head sources. This sensor was used to detect children. A voice generation device was used optionally to create speech phrases such as ‘hello there’, ‘where are you’, ‘can't see you’, depending on its sensory input (e.g. whether a child was detected or not). The speech was used purely to add variety to the robot's behaviour.

Figure 11

(a) A child with autism playing a chasing game. The boy went down on his knees, which gives him a better position facing the robot. (b) Another child decided to lie down on the floor and let the robot approach, resulting in turn-taking games. (c) A third child playing chasing games with a different mobile robot (Pekee, produced by Wany Robotics).

Figure 12

Playing turn-taking games with the robot. See text for a detailed description of this game.

Figure 13

(a) The Robota robot and (b, c) its modified appearances used in the trials with children with autism. The robot's main body contains the electronic boards and the motors that drive the arms, legs and head (Billard 2003). A pilot study showed the use of the robot's sensing abilities and autonomous behaviour was not suitable for our trials, thus the robot has since then been used as a remote-controlled puppet (controlled by the experimenter). A summary of the use of this robot in autism therapy and developmental psychology is provided by Billard et al. (in press). Experiments describing how children with autism react to different robot appearances are reported by Robins et al. (2004b,d).

Figure 14

Children with autism playing with a robot. (a) The picture shows a child interacting with Robota, the humanoid robot doll, playing an imitation game, whereby the robot can imitate arm movements when the child is sitting opposite the robot facing it and moving its arms in explicit ways that can be recognized by the robot (Dautenhahn & Billard 2002). Shown are the robot, the child and a carer providing encouragement. This was the first trial using Robota for playing with children with autism. Owing to the constrained nature of the set-up, required by the limitations of current robotic technology, we decided not to use this robot any longer as an autonomously operating machine, and later only used the robot remotely controlled by the experimenter (out of the children's sight). In one of the experiments, we varied the appearance of the robot and found that the children's initial response towards a ‘plain-looking’ robot is more interactive than towards the robot with its doll face (Robins et al. 2004b; figure 13). (b) The picture shows a child with autism playing imitation games with the robot. Note the completely unconstrained nature of the interactions, i.e. the child himself had decided to move towards and face the robot, on his own terms, after he had become familiar with the robot (as part of a longitudinal study; Robins et al. 2004c, 2005b). (c) The picture shows an autistic child playing with a mobile robot (Pekee, produced by Wany Robotics). The advantage of small mobile robots is that it allows the children to move around freely and adopt different positions, e.g. lying on the floor, kneeling, walking, even stepping over the robot, etc., exploring the three-dimensional space of potential interactions. In experiments with typically developing children, we developed a technique that can classify interactions of the children with the robot, using clustering techniques on the robot's sensor data. We were able to identify different play patterns that could be linked to some general activity profiles of the children (e.g. bold, shy, etc.; Salter et al. 2004). We will use this technique in the future to allow the robot to adapt to the child during the interactions (Salter et al. 2006; Franc¸ois et al. in press). In principle, this technique might also be used to assess the children's play levels or, possibly, for diagnostic purposes.

Figure 15

Varieties of interactions in playful encounters of children with autism and a mobile robot. Note the embodied nature of the interactions: children are using a variety of different postures and movements in order to play with the robot.

Figure 16

(a) Toy truck on the left and Labo-1 robot on the right. (b) Child interacting with the toy truck and (c) the robot. This comparative study involved 17 children with autism between 6 and 9 years old. Trials lasted approximately 10 min, i.e. the children interacted with the robot for 4 min. Next, both toy truck and robot were present for 2 min (robot switched off). Then, children played with the toy truck for 4 min. The order of presenting the toy truck and the robot was randomized. The toy truck (robot) was hidden during interactions with the robot (toy truck). Owing to the nature of our approach, we stopped a trial when a child seemed to become bored, distressed or wanted to leave the room. Interactions with the toy truck involved a lot of repetitive behaviour, e.g. spinning the wheels, pushing it against a wall; all children very quickly lost interest. Interactions with the robot were much more ‘lively’, the children were more engaged and played with the robot longer than with the toy truck. Note that any statements we make about the engagement of the children have been confirmed by teachers, carers and autism experts watching the videos with us. Differences were confirmed in behavioural coding of attention and eye gaze in both conditions. Details of this work are reported by Werry (2003) and Werry & Dautenhahn (in press).

Figure 17

The robot as a mediator. Top: examples of children with autism interacting with the experimenter, with the robot acting as a mediator (Robins et al. 2005b). Middle and bottom: the robot as a mediator facilitating interactions between children with autism. These examples emphasize the ultimate goal of the Aurora project, namely to help children with autism to connect to the social world of humans, and not necessarily to bond with robots.

Figure 18

A pair of children with autism simultaneously playing with the Labo-1 robot.