Genetic changes linked to two different syndromic forms of autism enhance reinforcement learning in early adolescent male but not female mice

Abstract

Autism Spectrum Disorder (ASD) is characterized by restricted and repetitive behaviors and social differences, both of which may manifest, in part, from underlying differences in corticostriatal circuits and reinforcement learning. Here, we investigated reinforcement learning in developing mice with mutations in either Tsc2 or Shank3, both high-confidence ASD risk genes associated with major syndromic forms of ASD. Using an odor-based two-alternative forced choice (2AFC) task, we tested early adolescent mice of both sexes and found male Tsc2 and Shank3B heterozygote (Het) mice showed enhanced learning performance compared to their wild type (WT) siblings. No gain of function was observed in females. Using a novel reinforcement learning (RL) based computational model to infer learning rate as well as policy-level task engagement and disengagement, we found that the gain of function in males was driven by an enhanced positive learning rate in both Tsc2 and Shank3B Het mice. The gain of function in Het males was absent when mice were trained with a probabilistic reward schedule. These findings in two ASD mouse models reveal a convergent learning phenotype that shows similar sensitivity to sex and environmental uncertainty. These data can inform our understanding of both strengths and challenges associated with autism, while providing further evidence that sex and experience of uncertainty modulate autism-related phenotypes.

Figure 1:. Early adolescent male, but not female Tsc2 Het mice show a gain of function in learning in early training in an odor based 2AFC task.

(A) Timeline of training. One week after weaning, male and female Tsc2 Het and WT animals were habituated to the chamber and ports through a shaping procedure in which they first learned to poke into the center port, followed by the peripheral ports, to receive water rewards. Here we used a deterministic reward schedule, where every correct choice led to a water reward. Odor cues were not present during habituation. At P30-P32 mice began session 1 of the odor-based 2AFC task. (B) In this task, mice self-initiate a trial by nose poke into the center port, which releases one of two odor cues. Bilateral “Go” lights then indicate the availability of reward, prompting mice to choose a peripheral side port. Choosing the correct port for that odor, results in a 2μL water reward. (C-D) Performance in male Tsc2 Het mice was significantly greater than WT males in both the first and second session of odor learning. This was captured by a significant main effect of genotype in both sessions and a genotype × time interaction in session 1. Significant Tukey post-hoc tests between Het and WT for each quartile are indicated by asterisks. (E-F) We also tested female Tsc2 Het and WT mice in the same odor-based 2AFC task on the same schedule. Female Tsc2 Het and WT did not show differences in learning performance for Session 1 or Session 2. Full statistical results are reported in the main text. Male Tsc2 Het, n = 15; WT, n = 13; Female Tsc2 Het, n = 10; WT, n = 10. Learning curves show mean for each quartile of all trials performed by individual mice ± SEM. *p<0.05, **p<0.01.

Figure 2:. Early adolescent male, but not female, Tsc2 Het mice have a higher learning rate when fit with the winning hybrid model.

We examined a series of models that use a reinforcement learning (RL) framework to account for differences in choice behavior between genotypes in Session 1 (see Materials & Methods). (A) Several of the candidate models that were tested were mixture models that combined a standard Q-learning RL model, representing an engaged state, with a biased model, representing a disengaged state, to account for periods of task disengagement at the policy level. The transition probabilities of these separate policy states are governed by a hidden Markov model. (B) Using Bayesian Information Criterion (BIC), we determined that models with the mixture policy (named hybrid here) had a lower BIC score. The change in BIC is calculated by the difference in each model’s BIC and the mean BIC of all models examined. All hybrid models were comparable, but the model with the lowest BIC (a0b1s_hybrid, blue box) contained a single α positive learning rate, a single inverse temperature parameter, , and a single strategy parameter,ξ₁, that indicated the likelihood for an animal to stick with their previous choice if they had received a reward. The full winning model is written in the schematic in (A). (C) We generated data by simulating the model with parameters fit on individual sessions. Then, we inspected the group similarity between simulated data (blue dotted line for male Tsc2 Het mice and green dotted line for female Tsc2 Het mice) and found that the model simulated with fit model parameters captures averaged mouse learning curve data. (D-E) Examples of individual trial-by-trial model fits for both male (D) and female mice (E). The blue for males (D) and the green for females (E) indicate the fraction correct performance for session 1 for two example animals from each sex. The lighter, transparent line indicates the simulated data from the winning hybrid model and the thin black dashed line represents the inferred probability that the mouse is in the latent “engaged” state, i.e. following the RL policy, as opposed to the biased state. (F) There was no significant difference in normalized bias between male Het and WT groups. (G) There was a significant difference between Tsc2 Het and WT males in learning rate α⁺ (plotted in log scale for visibility), Mann-Whitney test, U =54, p = 0.04, as well as α⁺ * , a post-hoc variable that captures the overall impact of an outcome on the policy, (I) Mann-Whitney test, U = 41, p = 0.008. There was no difference in the parameter (H) in males. In females, there were no differences between Tsc2 Het and WT mice in normalized bias (J), log(α)⁺ (K), (L), or log(α^+*) (M). Full statistical results are reported in the main text. Error bars represent SEM. *p<0.05, **p<0.01.

Figure 3:. Developing male, but not female Shank3B Het mice show a gain of function in learning and a higher learning rate during early training in an odor based 2AFC task.

(A) Male Shank3B Het showed stronger learning performance than WT littermates in the first training session, Session 1. This was captured by a significant main effect of time and genotype × time interaction. A Tukey post-hoc comparison test revealed a significant difference between Het and WT performance in the fourth quartile, indicated by the asterisk (p = 0.01). (B) There was a main effect of time but no genotype or genotype × time interaction in Session 2. We also tested female Shank3B Het and WT mice in the same odor-based 2AFC task on the same schedule. (C-D) Female Het and WT mice did not differ in learning performance for Session 1 or for Session 2 as evidenced by lack of significant main effect of genotype or an interaction. We next fit computational models to Shank3B male and female early learning data. Male Shank3B Hets and WT did not differ in normalized bias parameter (E), but did differ significantly in α⁺ (F) (plotted in log scale for visibility), (G), and the α⁺* variable (H). Female Shank3B Hets and WT did not differ significantly for any modeling parameter: (I) normalized bias parameter, (J) α⁺, (K) , (L) α^+* . Full statistical results are reported in the main text. Male Shank3B Het, n = 12; Male Shank3B WT, n = 13; Female Shank3B Het, n = 11; Female Shank3B WT, n = 11. Error bars represent SEM. *p<0.05, **p<0.01.

Figure 4:. Gain of function in early adolescent male Tsc2 and Shank3B Het is no longer apparent when reward schedule is probabilistic.

We tested male Tsc2 (A-C) and male Shank3B (D-F) in a probabilistic version of the odor-based 2AFC task where 10–20% of correct choices in session 1 and 2 were not rewarded (compared to Figures 1–3 data where 100% of correct choices were rewarded). Male Tsc2 Het and WT showed comparable learning performance in this probabilistic context in both session 1 and 2 (A,B). ITI speeds were also comparable (C) between Day 1 (Het, n = 8; WT, n = 9) and Day 2 (Het, n = 10; WT, n = 9). (D,E) Similarly, in male Shank3B mice, Het and WT learning performance was comparable between genotypes for both sessions. (F) There was also no significant difference between groups in ITI for either Day 1 (Het, n = 11; WT, n = 10) or Day 2 (Het, n = 11; WT, n = 7). We next fit our series of models to probabilistic learning trial-by-trial data. We found that there were no significant differences between model parameters (G-J) in male Tsc2 Het and WT mice nor between (K-N) male Shank3B Het and WT mice when performing in the probabilistic task context. Full statistical results are reported in the main text. Tsc2 Het, n = 10; Tsc2 WT, n = 10; Shank3B Het, n = 11; Shank3B WT, n = 11. Error bars show SEM. Violin plots show quartiles (25% and 75%), mean, and range.

Figure 5:. Early adolescent male Shank3B KO mice exhibit bimodal levels of trial completion and the more active subgroup shows gain of function in learning performance.

(A-B) In Session 1 and Session 2 the full group of male Shank3B KO mice (n = 13) had comparable learning performance to male Shank3B WT animals (n = 14). (C,D) Looking at total trials completed in each session, we noted that the total number of completed trials was bimodal for KO mice. Total trials completed were comparable to WT and Het siblings for one subgroup of KO (n = 5) (Sg1), but there was also a second subgroup (Sg2) (n = 8) that completed substantially fewer trials per session. Box and whisker plots show 25%, mean, 75%, and range with individual points indicating total number of trials for each individual mouse. Examining these groups more closely, revealed that (F) Sg1 weighed significantly more than Sg2, but (E) weight was not correlated with fraction correct performance in either subgroup. Based on these differences we decided to split the KO group into two subgroups and compare their learning to WT. (G-H) Sg1 KOs, who completed a typical range of trials, showed significantly better learning performance than WT in the odor based 2AFC in both Session 1 and Session 2. Sg2 KOs, who completed substantially fewer trials, performed at chance levels, significantly lower than WT in both sessions. Asterisks indicate post-hoc comparisons of subgroups to WT. In Session 1, quartile one, Sg2 KOs had significantly lower performance than WT (p = 0.03) and in quartile four, Sg1 KOs had significantly higher performance than WT (p = 0.002). In Session 2, Sg2 KOs had significantly lower performance than WT in quartile two (p = 0.01), three (p = 0.007), and four (p = 0.002). Sg1 outperformed WT in quartile three (p = 0.04). (I-J) Examination of intertrial interval (ITI) movement times showed that Sg2 was also significantly slower to move between ports than Sg1 in both sessions and that Sg1 ITI movement times more closely resembled WT mice, including in their correlation with fraction correct performance (K). Violin plots show quartiles (25% and 75%), mean, and range. Error bars represent SEM. *p<0.05, **p<0.01, ***p<0.001, ****p<0.0001.