Computational modeling of interventions for developmental disorders

Abstract

We evaluate the potential of connectionist models of developmental disorders to offer insights into the efficacy of interventions. Based on a range of computational simulation results, we assess factors that influence the effectiveness of interventions for reading, language, and other cognitive developmental disorders. The analysis provides a level of mechanistic detail that is generally lacking in behavioral approaches to intervention. We review an extended program of modeling work in four sections. In the first, we consider long-term outcomes and the possibility of compensated outcomes and resolution of early delays. In the second section, we address methods to remediate atypical development in a single network. In the third section, we address interventions to encourage compensation via alternative pathways. In the final section, we consider the key issue of individual differences in response to intervention. Together with advances in understanding the neural basis of developmental disorders and neural responses to training, formal computational approaches can spur theoretical progress to narrow the gap between the theory and practice of intervention. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Figure 1

Simulation of typical and atypical past tense acquisition predicting long-term compensated outcomes. (a) Empirical data (per cent accuracy) for typically developing children from Thomas et al. (2001) for a group of typically developing children on a past tense elicitation task for regular verbs, irregular verbs, novel verbs, and overgeneralization errors; and for a group of children with DLD from van der Lely and Ullman (2001), using the same elicitation task. Error bars show standard error of the mean. (b) Simulation data from Thomas (2005) for a connectionist past tense model, either in a typical condition or an atypical condition where the discrimination of the simple processing units was reduced by lowering the temperature of the sigmoid activation function (1 → 0.25). Model data are shown at a point that approximately matched the performance of the children (250 epochs of training). (c) Simulation data for the projected ‘adult’ outcome of typical and atypical trajectories (5000 epochs of training). The projected adult model reached ceiling on the training set but retained atypical generalization. Error bars show standard error over 10 replications with different initial random seeds.

Figure 2

Simulation of resolution of early delay. Group averaged developmental trajectories for 1000 simulated children in a model of English past tense formation, assuming a polygenic model for language delay (Thomas & Knowland, 2014). Delay was defined at Time 1 as networks whose performance fell more than 1 standard deviation below the population mean. Networks were defined as having resolving delay if their performance fell within this normal range by Time 5; and as having persisting delay if their performance remained below the normal range by Time 5 (see Thomas & Knowland, 2014, for further details). Error bars show standard deviations.

Figure 3

Individual differences in response to an enrichment intervention. Plot shows the relationship between treatment effects (change in proportion correct assessed at end of training) and the quality of the early environment for each simulated child (varying between 0 and 1) for (a) regular and (b) irregular verbs. Poorer family language environment predicted a larger treatment effect. This effect reduced for interventions later in development, and more so for irregular verbs. Early enrichment = 50 epochs, Late = 250 epochs, treatment effects assessed at 1000 epochs. Linear fits are shown for all conditions. Early enrichment for regular verbs was better fit by a log function (R² = .87), whereas linear functions explained more variance for the other three conditions.

Figure 4

The interaction of processing deficits with richness of early language environment. The plot depicts population performance on regular verbs early in development (50 epochs), split by individuals in impoverished or enriched environments, and stratified by individuals with different unit discriminability (temperature values 0.5–1.5). Interaction effect was at trend level (p = .06). Error bars show standard deviations.

Figure 5

Network architecture and problem domain for a model designed to explore how bespoke intervention sets can support learning in systems with atypical properties, in this case reduced connectivity: (a) network architecture; (b) example categorization problem, with 10,000 data points; the network is required to learn the category boundaries; (c) the training set given to the network, sufficient to learn the problem under typical conditions; (d) an example intervention set added to the training set to aid development under atypical conditions. Networks had 50 internal units (backpropagation network; learning rate = .1, momentum = .3, temperature = 1)

Figure 6

Developmental trajectories and internal representations in a typical case (TD), an atypical case with low connectivity (30%, C = 0.3) and the same atypical case experiencing an intervention. Top panel: Developmental trajectories; intervention commenced at 100 epochs. The intervention set was added to the training set for the duration of training. Vertical lines show epochs at which snapshots were taken. Lower panels: snapshots of the activation pattern of the unit for output category 2 in the three cases, which should respond only to the central band (see Figure 6). Hot colors represent more activity. (Fedor et al., 2013).

Figure 7

A model comparing interventions to remediate weaknesses or to improve strengths. (a) Developmental trajectories for naming and comprehension in a model acquiring the meanings (semantics) and word names (phonology) of 400 vocabulary items (averaged over 3 replications). The typical model shows the usual comprehension-production asymmetry. In the Word-Finding Difficulty (WFD) model, there was a restriction in the capacity of the pathway linking semantics to phonology (from 175 to 70 hidden units), which impacted on the development of naming, while comprehension trajectories did not reliably differ from normal. (b) Early intervention targeting the naming pathway (weakness). (c) Early intervention targeting the development of the phonological representations, the semantic representations, or both (strengths). (d) An intervention combining training on strengths and weakness. Intervention comprised training at five times the frequency on acquisition of these representations compared with naming and comprehension, beginning at 100 epochs and lasting for 100 epochs, shown by the shaded area. (Alireza et al., 2017).

Figure 8

A behavioral intervention to alter computational properties, in this case, to protect against overpruning of connectivity. (a) Performance of a group of 9 networks with a disorder caused by greater-than-usual loss of connectivity (red [dark gray]), compared with control networks (blue [middle gray]). Also shown are the disorder networks following an early behavioral intervention (green [light gray]), lasting between epochs 30 and 70. Effects of the intervention sustain to the end of development. (b) The number of network connections for the disorder group in untreated and intervention conditions. The intervention caused initial acceleration of loss but final preservation of a greater proportion of connections, associated with improved computational power. Midtraining = 250 epochs; End of training = 1000 epochs.

Figure 9

Different computational deficits producing the same behavioral impairment respond differently to intervention. Data show treatment effects of phonological versus semantic interventions for the Best et al. (2015) model of word-finding difficulties, where equivalent behavioral impairments were caused by three different underlying computational deficits. The atypical language profiles of two individual children were simulated and then interventions applied (here measured in how much naming development was advanced). The profile of each child was simulated either by reduced network connectivity (Deficit C), reduced hidden units (Deficit H), or a shallower sigmoid activation function in the artificial neurons (Deficit T). Intervention responses differed depending on how the deficit was implemented. Error bars show standard errors of 10 replications of each intervention (See Best et al., 2015, for further details).

Figure 10

Individual differences in response to intervention, following two types of intervention. Developmental deficits were caused by an overpruning disorder (Davis, 2017). The x axis shows treatment effect in terms of change in proportion correct. (a) Distribution for performance on the training set following the normalization or compensation treatment; (b) distribution for performance on the generalization set following either normalization or compensation treatment. [Population of 1000 networks, intervention for duration of 40 epochs applied early in development, epoch 30 of a life span of 1000, performance tested at 100 epochs].