Probabilistic reinforcement learning in adults with autism spectrum disorders

Abstract

Background: Autism spectrum disorders (ASDs) can be conceptualized as disorders of learning, however there have been few experimental studies taking this perspective.

Methods: We examined the probabilistic reinforcement learning performance of 28 adults with ASDs and 30 typically developing adults on a task requiring learning relationships between three stimulus pairs consisting of Japanese characters with feedback that was valid with different probabilities (80%, 70%, and 60%). Both univariate and Bayesian state-space data analytic methods were employed. Hypotheses were based on the extant literature as well as on neurobiological and computational models of reinforcement learning.

Results: Both groups learned the task after training. However, there were group differences in early learning in the first task block where individuals with ASDs acquired the most frequently accurately reinforced stimulus pair (80%) comparably to typically developing individuals; exhibited poorer acquisition of the less frequently reinforced 70% pair as assessed by state-space learning curves; and outperformed typically developing individuals on the near chance (60%) pair. Individuals with ASDs also demonstrated deficits in using positive feedback to exploit rewarded choices.

Conclusions: Results support the contention that individuals with ASDs are slower learners. Based on neurobiology and on the results of computational modeling, one interpretation of this pattern of findings is that impairments are related to deficits in flexible updating of reinforcement history as mediated by the orbito-frontal cortex, with spared functioning of the basal ganglia. This hypothesis about the pathophysiology of learning in ASDs can be tested using functional magnetic resonance imaging.

Figure 1

The PS task. Example stimulus pairs for the probabilistic stimulus selection (PS) task, which minimize explicit verbal encoding. The task consists of two phases. During the training phase, subjects are presented with three stimulus pairs (AB, CD, and EF). Each pair is presented separately in different trials in random order, and participants have to select among the two stimuli; correct choices are determined probabilistically. The frequency of positive and negative feedback for each stimulus is shown. Once a subject was able to score better than chance on AB and CD trials or completed 360 total trials, they proceeded to the test phase. In the test phase, 12 new pairs (only eight are shown) created from all unused combinations of training stimuli, are introduced and tested along with the three training pairs.

Figure 2

Early learning on the PS task-univariate analysis. Univariate analysis performance of 58 subjects (28 ASDs and 30 TYPs) during the first training block of the PSS task. There was no significant difference between the two groups for the AB and CD training pairs, but the ASDs performed significantly better (P = 0.017) than TYPs on the EF pair which is only correctly reinforced 60% of the time.

Figure 3

State–space learning curves for all trial types for ASD and TYP in block 1. The state–space model showing the performance on the three training pairs (AB, CD, and EF) for 58 subjects (28 ASDs and 30 TYPs) during the first training block. The bottom panel shows the exact trials for which performance was significantly different for the groups as places where the gray region is above or below the x axis. There was a greater overall probability of having better performance on CD trials if one was in the typically developing group, and an overall probability of having better performance on EF trials if one was in the ASD group.

Figure 4

Win–stay and lose shift behavior on the PS task in block 1. The win–stay and lose–shift percentages for 58 subjects (28 ASDs and 30 TYPs) during the first training block of the PSS task. The win–stay percentages were calculated by summing all incidents in which a subject chose the same stimulus (“stayed”) after receiving positive feedback (“winning”) for a given train pair and dividing it by the total number of times they received positive feedback, regardless of whether the feedback was accurate. Likewise, the lose–shift percentages were calculated by summing all incidents in which a subject chose a different stimulus (“shifted”) after receiving negative feedback (“losing”) for a given training pair and dividing it by the total number of times they received negative feedback. In Block 1, TYPs were significantly more likely than ASDs to win and stay; however, lose–shift performance was equivalent.