Interactive mirrOring Games wIth sOCial rObot (IOGIOCO): a pilot study on the use of intransitive gestures in a sample of Italian preschool children with autism spectrum disorder

Abstract

Background: Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by persistent deficits in social communication, social interaction, and restricted behaviors. The importance of early intervention has been widely demonstrated, and developmental trajectories in ASD emphasize the importance of nonverbal communication, such as intransitive gesture production, as a possible positive prognostic factor for language development. The use of technological tools in the therapy of individuals with ASD has also become increasingly important due to their higher engagement and responsiveness to technological objects, such as robots.

Materials and methods: We developed a training protocol using the humanoid robot NAO, called IOGIOCO (Interactive mirroring Games wIth sOCial rObot), based on the use of intransitive gestures embedded in naturalistic dialogues, stimulating a triadic interaction between child, robot and therapist. The training was divided into six levels; the first 2 levels were called "familiarization levels," and the other 4 were "training levels". The technological setup includes different complexity levels, from mirroring tasks to building spontaneous interactions. We tested the protocol on 10 preschool children with ASD (aged 2-6 years) for 14 weeks. We assessed them at recruitment (T0), at the end of training (T1), and after 6 months (T2).

Results: We demonstrated the tolerability of the protocol. We found that one group (n=4, males and 2 females) reached the training level, while another and group (n=6 males) remained at a familiarization level (mirroring), we analyzed the results for the two groups. In the group that reached the training levels, we found promising results, such as an improvement in the Social Adaptive Domain of the ABAS-II questionnaire between T0 and T2.

Conclusion: While current results will need a Randomized Controlled Trial to be confirmed, the present work sets an important milestone in using social robots for ASD treatment, aimed at impacting social and communication skills in everyday life.

Keywords: autism spectrum disorders; children; communicative gestures; social robot; technological rehabilitation.

Figure 1

Positions of the participants (adult-child-robot) and of the motion tracking system (Kinect) (A) and image of the clinical setting (B). Image of the participants’ positions as reported in (59), reproduced with the authors’ permission. Image of the room reproduced with permission from IRCCS Fondazione Don Gnocchi of Milan (Italy).

Figure 2

Block diagram of the entire system. The Kinect camera acquires the data. This data is constituted by the 3D joint positions’ of each detected person, and it is called skeleton. The skeleton is passed both to the Imitation System and the Gesture Recognition System, which are responsible for real-time mirroring and identification of the gesture mirrored, respectively. Through the ALMotion and ALAudioDevice modules, the two systems pass the joint angles and the feedback to the robot.

Figure 3

Pictures of NAO doing the selected gestures. Images of the robot reproduced with permission from IRCCS Fondazione Don Gnocchi of Milan (Italy) and Politecnico of Milan (Italy).

Figure 4

Block diagrams of the training modes for Levels 3–4. (A) Adult-Coach protocol and (B) Robot-Coach protocol. Image adapted from (59), with the permission of the authors.

Figure 5

Clinical protocol design.

Figure 6

ABAS II Results at different time points (T0-T1-T2) in Familiarization (FAM) and Training (TRAIN) groups for the Conceptual Adaptive Domain (CAD) (A), General Adaptive Composite Score (GAC) (B), Practical Adaptive Domain (PAD) (C), Social Adaptive Domain (SAD) (D). The purple dots represent each of the children, and the dashed lines are an approximation of their evolution.

Figure 7

Results at different time points (T0-T1-T2) in Familiarization (FAM) and Training (TRAIN) groups for the (A) Griffiths III- B scale, (B) Griffiths III – General Quotient, (C) McArthur – Number of Gestures.