Digitized Autism Observation Diagnostic Schedule: Social Interactions beyond the Limits of the Naked Eye

Abstract

The complexity and non-linear dynamics of socio-motor phenomena underlying social interactions are often missed by observation methods that attempt to capture, describe, and rate the exchange in real time. Unknowingly to the rater, socio-motor behaviors of a dyad exert mutual influence over each other through subliminal mirroring and shared cohesiveness that escape the naked eye. Implicit in these ratings nonetheless is the assumption that the other participant of the social dyad has an identical nervous system as that of the interlocutor, and that sensory-motor information is processed similarly by both agents' brains. What happens when this is not the case? We here use the Autism Diagnostic Observation Schedule (ADOS) to formally study social dyadic interactions, at the macro- and micro-level of behaviors, by combining observation with digital data from wearables. We find that integrating subjective and objective data reveals fundamentally new ways to improve standard clinical tools, even to differentiate females from males using the digital version of the test. More generally, this work offers a way to turn a traditional, gold-standard clinical instrument into an objective outcome measure of human social behaviors and treatment effectiveness.

Keywords: autism; digital biomarkers; network connectivity; non-linear complex dynamics; social dyads; socio-motor parameters; stochastic analysis; time series analysis; wearables.

Figure 1

Leveraging the wearable sensors revolution to significantly augment traditional pencil and paper methods and advance the behavioral sciences. (A) The macro-level of behavioral description inevitably misses fast and subtle information in dyadic social interactions, as information flows between the Central and the Peripheral Nervous Systems (the CNS and the PNS). (B) Micro-level coherence information, automatic social mirroring, lead-lag patterns, and micro-gestures of the face and body, among other socio-motor behaviors, can only be captured with high grade instrumentation and proper analytics. (C) Macro- and micro-levels of inquiry can be integrated to advance social behavioral sciences, e.g., the Autism Diagnostic Observation Schedule (ADOS) is used as a backdrop experimental assay to probe dyadic social interactions, combined with wearables. to digitize standard clinical criteria. Light wearables embedded in the clothing unobtrusively and continuously co-register child and clinician (certified as rater) during the administration of the ADOS test to detect autism. Micro-fluctuations in the timeseries data from these wearables (e.g., acceleration waveforms) can be converted to standardized micro-movement spikes (MMS) to derive various socio-motor biometrics. (D) The Earth Mover’s Distance (EMD) metric is used to ascertain the pairwise difference between frequency histograms of MMS and build a matrix with entries capturing the dyadic interactions (circled entries) and the self-interactions of body nodes. (E) The averaged sum across all dyadic interaction entries per unit time, obtained task by task, in the order in which they were administered, gives us a session profile of the dyadic variability, as one example of several metrics used here to objectively quantify socio-motor behavior simultaneously at the macro- and micro-levels of inquiry.

Figure 2

Analytical pipeline and visualization tools. (A) Frequency domain power spectrum analysis of pairwise sensor data provides input to obtain cross coherence spectrum phase across frequencies. (B,C) A parameterization of the data into three corresponding matrices: MaxCoherence, MaxPhase, MaxFrequency. Each entry contains the maximal cross coherence value obtained pairwise across all sensors. For example, row 1 contains the values for the child’s right wrist (R) sensor in relation to all other sensors (child left wrist L, child torso T, clinician right wrist R, clinician left wrist L, and clinician torso T. Dyad entries of the matrix are at the right upper quadrant (child→clinician) and left lower quadrant (clinician→child). Self-interaction entries are at the left upper quadrant (child) and the right lower quadrant (clinician). Corresponding entries in the MaxPhase matrix reflect the phase value at which the pairwise coherence is maximal while the same entry in the MaxFrequency reflect the corresponding frequency value. (D) Adjacency matrix used to represent a dynamically changing weighted connected graph is obtained by retaining the entries at positive phase under convention i→j for the (i,j) entry. Dyadic interaction entries are circled. (E) Outdegree for each node denotes the number of links from that node to other nodes. (F–H) Visual tools and sample metrics to characterize socio-motor behaviors in social dyads. (F) Visualizing the network to track its states as they dynamically change in the Construction Task for a child-clinician dyad (EP01, with 17-unit times of 12 s time length for each sample under Independent Identically Distributed (IID) assumption, explained in Methods). The size of the node corresponds to the OutDegree value, the arrow’s color is the maximum cross-coherence value, the thickness is the phase value and the direction comes from the computation involving Outdegree (see methods.) (G) Sample profile over tasks of Experimental Participant 01, EP01 showing the COH1 and COH2 unfolding within the session. These terms are used to obtain a socio-motor metric (see Methods.) (H) Lead profile for the dyad involving EP01 and revealing the clinician prevalent leadership for most tasks, but Loneliness (LLN) and Creating a Story (CS), where the child leads (please see Supplementary Material Table S2 for task description and acronyms).

Figure 3

A measure of social readiness potential: Dyadic strength expressed in relation to ADOS sub-scores (Social Affect, (SA) and the so-called Repetitive Ritualistic Behaviors, (RRB)). (A) Dyad strength (see methods) tends to be higher in neurotypical controls, with a tendency towards lower (normalized) ADOS scores and some higher variations in the more ambiguous RRB sub-score. The trend in the autistic dyadic strength scores is towards lower values and higher ADOS score, yet 5/15, (33.3%) fall within the neurotypical lower range despite higher ADOS scores, thus signaling a hidden capacity for social dyadic exchange. (B) This information can be further unfolded for each participant, whereby a score of social readiness potential is obtained as a relative quantity measuring the absolute difference between each participant and the neurotypical with the highest dyad strength. Many of the autistic participants do have socio-motor strength in the dyadic interaction, despite high ADOS scores.

Figure 4

Mirroring Bias Effects. (A,B) Biased rating in the ADOS test quantified through mirroring metric of lead-lag social patterns of self-interactions (individual kinematic synergies) as percentage in leading patterns across the cohort. Neurotypical Controls and Clinician show broader range of values than Autistics the visit one, under the most appropriate module and same rater. (C) Mirroring effect and raters’ leading bias persist in the shared dyadic cohesiveness as the module and rater change. Visit one and three (appropriate module) under two different raters show a reduction in parameter range for autistics relative to controls. Visit two and four (less appropriate module and by then familiar rater) show an increase in parameter range for autistics relative to visits one and three.

Figure 5

Monologue style of ADOS test. Rater leads social interaction a large percentage of the time for each child. (A) For all 26 children divided into neurotypical controls and autistics on the x-axis and on the y-axis, the % of time (taken across all ADOS tasks) that the rater or the child leads the interaction. Clinician rater leads on average for each child, across all tasks. (B) Group data per visit showing the summary of the % time that the person leads the social interaction.

Figure 6

High sensitivity of micro-level metrics captures socio-motor changes over time and serves to measure rater’s reliability-style. (A–D) ADOS scoring system does not capture change in developmental socio-motor physiology. Across visits, the macro-level scores remain static, but the micro-level socio-motor parameters change. These include leading profile, Dyadic Variability and Dyadic Coherence. (E,F) High sensitivity to changing clinician rater prevails across all children and tasks. The self-interaction parameters (individual kinematic synergies) of autistic participants are sensitive (at the micro-level) to changing the rater. (G,H) Leading profiles in the same autistic cohort shift as the raters differ.

Figure 7

Vignettes samples show bottlenecks of pencil and paper observation methods. (A–D) Macro-level scoring system does not capture change in developmental socio-motor physiology. Experimental participants EP16 and EP21 are perceived very different by the same clinician in relation to leading patterns (A); Dyad Variability (B) and Dyad Coherence (C,D) in each of the clinician→child and child→clinician directions. (E–H) Ill-posed autism detection problem: Given the ADOS score, what is the most likely socio-motor phenotype? Each clinician perceives the same child differently across all micro-level socio-motor indexes.

Figure 8

Digital biomarker for personalized design of adaptive targeted therapy. (A) Parameter space spanned by dyad strength on x-axis and delay in cohesiveness (see text for details) on the y-axis. Ideally within a social interaction one would desire high dyad strength and a broad range of response cohesiveness, spanning from fast to slow. Note that most controls have high dyad strength and their responses vary broadly from lower to higher cohesiveness delay, whereas autistics are primarily in the region of low dyad strength and delayed cohesiveness (i.e., they synchronize their body biorhythms with those of the rater, but there is a larger lag than desirable.) (B) Task ranking criteria based on dyad strength calculated for various tasks from participants in visit one. Functional and Symbolic Imitation, FSI is assigned the highest rank (easiest task to perform) as it depicts the best dyad strength while PH having the lowest dyad strength is assigned the lowest rank (most difficult task to perform).

Figure 9

Digital ADOS automatically separates females from males in a random draw of the population. (A) Scatters and histograms of parameters derived from interconnected networks representing the dynamic dyad provide evidence for fundamental differences in connectivity across the nodes of the network composed by the child and clinician dyad. (B) The empirically estimated Gamma scale parameter (the noise to signal ratio, (NSR)) of the micro-movement spikes (MMS) derived from acceleration separate females from males with lower NSR in females and tighter distribution than males. The characteristic pathlength denoting average shortest distance path from each node to every other node of the network is significantly shorter in females than in males. The betweenness centrality is higher in females than males and the clustering coefficient is also higher in females. All differences are statistically significant with p << 0.01. (C) Summary graph for angular speed is consistent with the acceleration patterns. (D) Females show lower NSR, lower characteristic pathlength and higher betweenness centrality and lower clustering coefficient. All differences are statistically significant according to non-parametric rank-sum test p << 0.01.