Computer vision analysis captures atypical attention in toddlers with autism

Abstract

To demonstrate the capability of computer vision analysis to detect atypical orienting and attention behaviors in toddlers with autism spectrum disorder. One hundered and four toddlers of 16-31 months old (mean = 22) participated in this study. Twenty-two of the toddlers had autism spectrum disorder and 82 had typical development or developmental delay. Toddlers watched video stimuli on a tablet while the built-in camera recorded their head movement. Computer vision analysis measured participants' attention and orienting in response to name calls. Reliability of the computer vision analysis algorithm was tested against a human rater. Differences in behavior were analyzed between the autism spectrum disorder group and the comparison group. Reliability between computer vision analysis and human coding for orienting to name was excellent (intra-class coefficient 0.84, 95% confidence interval 0.67-0.91). Only 8% of toddlers with autism spectrum disorder oriented to name calling on >1 trial, compared to 63% of toddlers in the comparison group (p = 0.002). Mean latency to orient was significantly longer for toddlers with autism spectrum disorder (2.02 vs 1.06 s, p = 0.04). Sensitivity for autism spectrum disorder of atypical orienting was 96% and specificity was 38%. Older toddlers with autism spectrum disorder showed less attention to the videos overall (p = 0.03). Automated coding offers a reliable, quantitative method for detecting atypical social orienting and reduced sustained attention in toddlers with autism spectrum disorder.

Keywords: autism spectrum disorders; behavioral measurement; development; pre-school children; social cognition and social behavior.

Figure 1

Experimental setup and facial landmark detection: A) participant sitting on caregiver's lap with experimenter standing behind. A subset of the detected facial landmarks are shown demonstrating detection of face position (green dots). B) Example frames from video stimuli.

Figure 2

Example of detecting a head turn after a name call (time=0). From estimated yaw position (shown in blue circles), we detect instances when the head left the frame from a head turn (shown in green squares). Screenshots from the recorded video are displayed above.

Figure 3

Predicted means (lines) and 95% confidence intervals (shaded areas) for proportion of time engaged in the task from models covarying for the age by group interaction (p=0.03). The autism group (blue dashed line) showed less time attending at older ages than the comparison group (pink solid line).

Figure 4

Orienting to Name: (A) Proportion of participants orienting to name in comparison group (pink) vs autism group (blue) on any of the name calls (left panel) vs multiple (2 or 3 times, right panel, p=0.002), with error bars showing 95% CI (confidence intervals) of proportions. (B) Kaplan-Meier plots of cumulative events over time with 95% CI (shading) for each group showing slower orienting in the autism group (blue dashed line).