Quantifying interaction with robotic toys in pre-term and full-term infants

Abstract

Infants born pre-term are at an increased risk for developmental, behavioral, and motor delay and subsequent disability. When these problems are detected early, clinical intervention can be effective at improving functional outcomes. Current methods of early clinical assessment are resource intensive, require extensive training, and do not always capture infants' behavior in natural play environments. We developed the Play and Neuro Development Assessment (PANDA) Gym, an affordable, mechatronic, sensor-based play environment that can be used outside clinical settings to capture infant visual and motor behavior. Using a set of classification codes developed from the literature, we analyzed videos from 24 pre-term and full-term infants as they played with each of three robotic toys designed to elicit different types of interactions-a lion, an orangutan, and an elephant. We manually coded for frequency and duration of toy interactions such as kicking, grasping, touching, and gazing. Pre-term infants gazed at the toys with similar frequency as full-term infants, but infants born full-term physically engaged more frequently and for longer durations with the robotic toys than infants born pre-term. While we showed we could detect differences between full-term and pre-term infants, further work is needed to determine whether differences seen were primarily due to age, developmental delays, or a combination.

Keywords: infant development; movement; neurodevelopment; neurorehabilitation; preterm birth; rehabilitation robotics; toy; vision.

Figure 1

Selection of trials for analysis. Of the 34 infants in the pilot study, five were excluded due to procedural errors. Of the 29 testing sessions considered, 10 elephant videos, 11 orangutan videos, and 7 lion videos were deemed unusable and discarded from further analysis. In total, we evaluated 19 elephant trials, 18 orangutan trials, and 22 lion trials from 24 infants.

Figure 2

The PANDA Gym and sensor-based robotic hanging toys. The first prototype of the Gym had a 4” × 4” feet force sensing mat and two GoPro cameras with stereo views for top and side views of the infant at play. Early toy prototypes show three mechatronic toys–Orangutan, Lion, and Elephant. Each toy contains a 3-axis inertial measurement unit (IMU), force sensing resistors, custom pressure nose, LEDs and audio feedback. A sketch of the construction of the elephant toy.

Figure 3

Photographic examples of infant interactions with the three toys. (A) Involuntary: the infant’s gaze is not focused on the elephant toy but contacts it involuntarily with a foot. (B) Gaze: the infant is visually focused on the orangutan toy. (C) Hand touch, and (D) hand grasp: hand grasp involved a power grasp while hand touch involves a palmar contact; distinction was also made based on shape and angle of palm and fingers around the toy. Photos (E, F) are examples of a foot touch being a simple contact and foot kick a forceful one, respectively. In photo (E) the infant puts the orangutan toy in their mouth. Photo (H) shows an infant separating the hands of orangutan toy which had a reed-switch keeping the arms closed. In photo (I) the infant simultaneously touches, grasps, gazes at, and places the toy in its mouth. Note that all other photos in Figure 5 also represent overlapping interactions.

Figure 4

Photographic example of the video coding process. The infant is seen lying supine in the PANDA Gym with a two-minute video segment for analysis. MAXQDA software allowed the coder to assign relevant movement codes to appropriate areas along the video’s timeline. The figure above shows a simultaneous interaction and how that was coded for this infant.

Figure 5

Fourteen infants were born full-term births and nine were pre-term. At testing the average age and standard deviations of each group was Full-term: 5.73 ± 1.47 months and Pre-term: 6.36 ± 1.22 months (p = 0.33). In total, we evaluated 19 elephant trials, 18 orangutan trials, and 22 lion trials from 24 infants.

Figure 6

Frequency of infant interactions with the elephant toy. Full-term infant interactions are reported in the top row and pre-term infant interactions in the bottom row. Hand touch and hand grasp show most frequent interactions.

Figure 7

Frequency of infant interactions with the orangutan toy. Full-term infant interactions are reported in the top row and pre-term infant interactions in the bottom row. Mouth, hand touch and hand grasp show most frequent interactions. Some Full-term infants used their feet to interact with this toy.

Figure 8

Frequency of infant interactions with the lion toy. Full-term infant interactions are reported in the top row and pre-term infant interactions in the bottom row. Foot and kick show most frequent interactions.

Figure 9

Time duration (in seconds) of infant interaction with the three toys combined. When the voluntary feet and hand interactions are considered in terms of time spent, we see that the elephant and orangutan had most interactions.

Figure 10

Time duration (in seconds) of infant gaze interaction with the three toys. Pre-term infants were more likely to gaze at the orangutan toy than the elephant and lion. Short black horizontal lines represent median.