Cortical norepinephrine-astrocyte signaling critically mediates learned behavior

Abstract

Updating behavior based on feedback from the environment is a crucial means by which organisms learn and develop optimal behavioral strategies^1-3. Norepinephrine (NE) release from the locus coeruleus (LC) has been shown to mediate learned behaviors^4-6 such that in a task with graded stimulus uncertainty and performance, a high level of NE released after an unexpected outcome causes improvement in subsequent behavior⁷. Yet, how the transient activity of LC-NE neurons, lasting tens of milliseconds, influences behavior several seconds later, is unclear. Here, we show that NE acts directly on cortical astrocytes via Adra1a adrenergic receptors to elicit sustained increases in intracellular calcium. Chemogenetic blockade of astrocytic calcium elevation prevents the improvement in behavioral performance. NE-activated calcium invokes purinergic pathways in cortical astrocytes that signal to neurons; pathway-specific astrocyte gene expression is altered in mice trained on the task, and blocking neuronal adenosine-sensitive A1 receptors also prevents post-reinforcement behavioral gain. Finally, blocking either astrocyte calcium dynamics or A1 receptors alters encoding of the task in prefrontal cortex neurons, preventing the post-reinforcement change in discriminability of rewarded and unrewarded stimuli underlying behavioral improvement. Together, these data demonstrate that astrocytes, rather than indirectly reflecting neuronal drive, play a direct, instrumental role in representing task-relevant information and signaling to neurons to mediate a fundamental component of learning in the brain.

Extended Data Fig. 1.

Change in d-prime after hit, false alarm, and unreinforced trials (n = 18 mice). **, p < 0.01, two-tailed paired t-test. Data show mean ± SEM.

Extended Data Fig. 2.

A. Schematic of experimental design for LC axonal stimulation and astrocyte calcium imaging. B. Example 2-photon images of LC axons expressing ChR2 (blue) and astrocytes expressing GCaMP6f (green). Scale bar, left panel, 50 um; right panels, 20 um. C. Average astrocyte calcium responses following NE axon stimulation for 0.5, 1.0, and 2.0 seconds. Each gray line indicates data from one animal. Solid bars denote mean, error bars are s.e.m. (n=4 mice). D. Peak dF/F for astrocytes during hit, miss, correct rejection, and false alarm trials in go/no-go task. Each gray line indicates one astrocyte. Black line indicates population average, error bars are s.e.m. E. Mean dF/F for astrocytes. n = 148 astrocytes (n=3 mice). F. Left: Example FISH image of DAPI (blue), aquaporin 4 (Aqp4, red), and Adra1a (green) mRNA expression in the motor cortex. Scale bar: 100um. Right (top): Example astrocyte ROI (dashed line) with overlapping DAPI (blue) and Aqp4 (red) mRNA expression. Scale bar: 5um. Right (bottom): Same as right top, including Adra1a (green) mRNA expression. Scale bar: 5um G. Quantification of astrocytes (Aqp4+ cells) positive and negative for Adra1a transcripts. Each data point represents the fraction for one animal (n=3 mice). H. Quantification of Adra1a puncta on astrocytes. I. Top: Astrocyte and neuronal population dF/F on hit, miss, and correct rejection trials. Bottom: Rasters of neuronal dF/F on hit, miss and correct rejection trials. Astrocyte rasters are shown in text Fig. 1I. n = 148 astrocytes, 44 neurons (n=3 mice). J. Average mean dF/F for hit (left), miss (center), and correct rejection (right) trials for astrocytes and neurons. K. Average peak dF/F for hit, miss, and correct rejection trials for astrocytes and neurons. P values show comparisons based on Mann Whitney U-test in C, two-tailed paired t-test with Bonferroni correction in D and E, two-tailed unpaired t-test in J and K. *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean ± SEM.

Extended Data Fig. 3.

A. Average astrocyte dF/F on false alarm trials for chemogenetic silencing of LC-NE in saline (top) and CNO (bottom) sessions. n = 101 astrocytes, saline; n = 108 astrocytes, CNO (n=3 mice). B. Fraction of astrocytes responsive to false alarms in saline and CNO sessions. C. Average mean dFF for astrocytes in saline and CNO sessions. **, p < 0.01, two-tailed unpaired t-test. Data show mean ± SEM.

Extended Data Fig. 4.

A. Average astrocyte dF/F on false alarm trials without manipulations (blue) and with knockdown of Adra1a receptors (red) sessions; population average (top) and individual cells (bottom). n = 148 astrocytes with no manipulation; n = 44 astrocytes with knock down of Adra1a (n=3 mice). B. Average mean dF/F for astrocytes in no manipulation and knockdown of Adra1a sessions. C. Average peak dF/F for astrocytes in no manipulation and knockdown of Adra1a sessions. D. Average FA response duration for responsive astrocytes in no manipulation and knockdown of Adra1a sessions. E. Fraction of astrocytes responsive to false alarms in no manipulation and knockdown of Adra1a sessions. ***, p < 0.001, two-tailed unpaired t-test.

Extended Data Fig. 5.

A. Average astrocyte dF/F on hit trials for chemogenetic inactivation of astrocytes in saline (gray) and CNO (orange) sessions; population average (top) and individual cells (bottom). n = 160 astrocytes, saline; n = 99 astrocytes, CNO (n = 5 mice). B. Same as A on miss trials. C. Same as A on correct rejection trials. D. Average mean dF/F for astrocytes in saline and CNO sessions for, from left to right: hit, miss, correct rejection, and false alarm trials. E. Average peak dF/F for astrocytes in saline and CNO sessions, from left to right: hit, miss, and correct rejection trials. P values show comparisons based on two-tailed unpaired t-test in D and E. *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean ± SEM.

Extended Data Fig. 6.

Change in d-prime after hit, false alarm, and unreinforced trials for saline (left) and CNO (right) sessions in astrocyte Gq-DREADD expressing animals (n = 12 mice). *, p < 0.05, one-tailed paired t-test. Data show mean ± SEM.

Extended Data Fig. 7.

A. Purity of isolated astrocytes was confirmed for both untrained and trained animals by measuring the expression of cell-type specific markers for astrocytes (Slc1a3, Slc1a2, Aqp4, Gfap, S100b), oligodendrocytes (Olig1, Olig2), microglia (Tmem119), and neurons (Map2, Tubb3). Gene expression was normalized to Gapdh. Bars represent mean ± SEM. B. Heat map of gene expression fold change for 40 selected ATP/ adenosine pathway genes. 15 genes that were significantly different in trained mice compared to untrained mice are shown in Fig. 3D. UN, untrained animals, T, trained animals; UN1-4 and T1-6 represent independent hemispheres.

Extended Data Fig. 8.

Change in d-prime after hit, false alarm, and unreinforced trials for saline (left) and DPCPX (right) sessions (n = 3 mice). *, p < 0.05, one-tailed paired t-test. Data show mean ± SEM.

Extended Data Fig. 9.

A. Average neuron firing rate difference between go and no-go trials after hit, false alarm or unreinforced trials for saline (left), CNO (center), and DPCPX (right) sessions. Saline, n = 377 neurons (n = 4 mice); CNO, n = 271 neurons (n = 3 mice); DPCPX, n = 186 neurons (n=3 mice). B. Average neuron firing rate for go and no-go stimuli following unreinforced trials in saline (left), CNO (center) and DPCPX (right) sessions. C. Same as B following hit trials.

Fig 1.. Astrocytes exhibit sustained increases in intracellular calcium following a false alarm in a reinforcement learning task.

(A) Task design. (B) Schematic of task and timing/magnitude of LC-NE neuronal activity. ITI, inter-trial interval. (C) Example psychometric curves from one mouse showing probability of pressing the lever by tone intensity for all trials (black) and on trials following a false alarm (red). (D) Change in false alarm rate, hit rate, and d-prime on trials following a false alarm, calculated after subtracting shuffled data (n=18 mice). P values show comparisons based on two-tailed paired t-test. (E) Methods for dual imaging of astrocytes and neurons. (F) Example 2-photon image of astrocytes and neurons expressing GCaMP6f and jRGECO1a, respectively. Scale bar = 50 μm (G) Mean dF/F of three example astrocytes aligned to tone onset for false alarm (FA), correct rejection (CR), miss, and hit trials. (H) Session averaged dF/F for the three example astrocytes aligned to tone onset, by trial type. (I) Top: raster plot showing average astrocyte dF/F by trial type, aligned to the time of tone inset. Bottom: average dF/F for all astrocyte ROIs by trial type. n = 148 astrocytes (n = 3 mice). (J) Top: raster plot showing average neuronal dF/F on false alarm trials. Bottom: Population average dF/F for all neurons and all astrocytes (from I) on false alarm trials, aligned to time of tone onset. Gray zones depict baseline +/− s.d. n = 44 neurons (n = 3 mice) (K) Peak dF/F for astrocytes and neurons in response to a false alarm. (L) Duration of astrocyte and neuronal responses to a false alarm. (M) Fraction of astrocytes (left) and neurons (right) responsive to each trial type. P values show comparisons based on two-tailed unpaired t-test in K, L. *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean +/− SEM.

Figure 2.. NE and adrenergic receptor-specific responses of astrocytes and effects of manipulating calcium on behavior.

A. Experimental design for chemogenetic silencing of LC-NE neurons while imaging astrocyte calcium. B. Experimental timeline. C. Example trial averaged astrocyte trace on a false alarm trial for a saline and CNO session. D. Left: Average peak astrocyte dF/F on false alarm trials for saline and CNO sessions. Right: Average FA response duration for responsive astrocytes in saline and CNO sessions. CNO, n =108 astrocytes; saline, n = 101 astrocytes (n = 3 mice). P values show comparisons based on two-tailed unpaired t-test. E. Experimental design for imaging astrocyte calcium in astrocytes with Adra1a receptor knockdown. F. Example 2-photon image showing overlapping Cre and GCaMP6f expression in astrocytes. Scale bar, 50 um. G. Example traces for hit, miss, correct rejection, and false alarm trials from one astrocyte. H. Population averages for peak astrocyte dF/F for all trial types in Adra1a-R KD astrocytes. n = 44 astrocytes (n = 3 mice). P value determined using one-way ANOVA. I. Experimental design and timeline for chemogenetic manipulation of astrocyte calcium. J. Top: Average dF/F for each astrocyte ROI over the course of a saline imaging session. Bottom: population average. Red dots indicate false alarms. K. Average dF/F for each astrocyte ROI over the course of a CNO imaging session. Bottom: population average. Red dots indicate false alarms. L. Average astrocyte dF/F on false alarm trials for saline (top) and CNO (bottom) sessions. CNO, n = 99 astrocytes; saline, n = 160 astrocytes (n = 5 mice). M. Population average from L. N. Average peak dF/F for astrocytes in saline and CNO sessions. O. Average response duration for responsive astrocytes in saline and CNO sessions. P. Change in false alarms (FA), hits, and d-prime on trials after a false alarm for saline (top) and CNO (bottom) sessions (n= 12 mice). P values show comparisons based on two-tailed unpaired t-test in N, O, and two-tailed paired t-test in P. *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean ± SEM.

Figure 3.. Astrocytes mediate reinforcement learning via purinergic signaling.

A. Experimental design of adenosine fluorometric assay of astrocyte cultures. B. Adenosine assay showing an increase in adenosine release in response to NE (20 μM) from cultured astrocytes. (n = 3). NE-induced release of adenosine is blocked in the presence of prazosin hydrochloride. (n = 3). RFU = relative fluorescence units. C. Schematic for gene expression analysis. D. Average gene expression fold change of significantly different genes in trained mice, measured by qRT-PCR and normalized to untrained mice. (untrained mice, n = 4 hemispheres; trained mice, n = 6 hemispheres). E. Pie chart showing the functional classification of differentially expressed genes annotated to their GO molecular function and GO biological process terms. The size of each wedge represents the number of significantly changed genes in each functional class, as noted. F. (Left) Experimental design applying saline or DPCPX topically to prefrontal and motor cortex followed by behavioral testing. (Center, right) Change in false alarms, hits, and d-prime on trials after a false alarm in saline and DPCPX conditions. P values show comparisons based on two-tailed unpaired t-test in B, D, and two-tailed t-test in F (n=3 mice). *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean + SEM.

Figure 4.. Astrocyte calcium manipulations and A1 receptor blockers occlude improvements in neuronal stimulus encoding post- false alarm.

A. Experimental design for saline, CNO, and DPCPX recordings. B. Top: Rasters showing activity on single trials for go and no-go stimuli after unreinforced (gray) and false alarm (red) trials for example units. Bottom: Average activity for go/no-go stimuli after unreinforced and false alarm trials. C. Population averaged firing rate for go and no-go stimuli on trials after a false alarm in saline (top), CNO (center), and DPCPX (bottom) conditions. D. Quantification of the difference between go and no-go firing rates on trials after hit (blue), false alarm (red), and unreinforced (gray) trials. E. D-prime after false alarm versus d-prime after unreinforced trials for saline, CNO, and DPCPX conditions. Blue dots represent individual units. Regression lines are shown in red with 95% bootstrap CI zone in dark blue. Inset: quantification of d-prime post false alarm versus post-unreinforced trials. F. Quantification of d-prime after no reinforcement, false alarm, and hits for saline, CNO, and DPCPX sessions. n = 377 neurons, saline (n = 4 mice); n = 271 neurons, CNO (n = 3 mice); n = 186 neurons, DPCPX (n=3 mice). P values show comparisons based on two-tailed paired t-test in D and E (insets), on bootstrap with replacements in E (slopes), and on two-tailed unpaired t-test in F. *, p < 0.05; **, p < 0.01; ***, p < 0.001. Data show mean ± SEM. G. Model of the role of NE-astrocyte-adenosine signaling in learned behavior.