Simulation of cholinergic and noradrenergic modulation of behavior in uncertain environments

Abstract

Attention is a complex neurobiological process that involves rapidly and flexibly balancing sensory input and goal-directed predictions in response to environmental changes. The cholinergic and noradrenergic systems, which have been proposed to respond to expected and unexpected environmental uncertainty, respectively, play an important role in attention by differentially modulating activity in a multitude of cortical targets. Here we develop a model of an attention task that involves expected and unexpected uncertainty. The cholinergic and noradrenergic systems track this uncertainty and, in turn, influence cortical processing in five different, experimentally verified ways: (1) nicotinic enhancement of thalamocortical input, (2) muscarinic regulation of corticocortical feedback, (3) noradrenergic mediation of a network reset, (4) locus coeruleus (LC) activation of the basal forebrain (BF), and (5) cholinergic and noradrenergic balance between sensory input and frontal cortex predictions. Our results shed light on how the noradrenergic and cholinergic systems interact with each other and a distributed set of neural areas, and how this could lead to behavioral adaptation in the face of uncertainty.

Keywords: cholinergic; neuromodulation; noradrenergic; simulation; uncertainty.

Figure 1

Experimental set-up of the behavioral task. The experiment is modeled after a rodent attention task that dissociates expected uncertainty from unexpected uncertainty. A trial begins by placing a subject in the center of a ring of 36 lights. While the subject is in the center of the ring, a light is briefly flashed. If the subject detects the light, it goes to the location where it thought the light flashed, performs a nose poke, and returns to the center of the ring. If the subject poked its nose in the hole where the light was flashed it receives a reward when it returns to the center, otherwise it does not. The flashed light positions follow a Gaussian distribution whose standard deviation signifies the expected uncertainty. This is indicated in the figure by the filled in yellow circles, which signify the probability of a light flashing in a particular location (a larger circle indicates a higher probability of flashing). After a set number of trials a new Gaussian distribution of lights is presented and this shift of the mean and standard deviation signifies the unexpected uncertainty.

Figure 2

Neural architecture. The neural network contains a total of six groups (two neuromodulatory, three cortical, one input). The visual input group drives activity in the VC (visual cortex). VC and PFC (prefrontal cortex) provides input to the PPC. The noradrenergic system, originating in the LC (locus coeruleus), enhances the decay of the synaptic connections (“forgetting”) between VC and PFC, and PFC and PPC (posterior parietal cortex), as indicated by NA(^*) in the figure. The noradrenergic system also enhances the gain in the BF (basal forebrain) and the input to the PPC from VC [indicated by the NA(+)] and suppresses input to the PPC from the PFC [indicated by the NA(−)]. The cholinergic system, originating in the BF, enhances input to VC and PPC [indicated by the ACh(+)] and suppresses recurrent activity in the PFC and input to the PPC from the PFC [indicated by the ACh(−)].

Figure 3

Illustration of the changes in network activity due to different levels of ACh and NA. (A) ACh is low (low expected uncertainty). Recurrent connections within the PFC and inputs from the PFC to the PPC are enhanced. Visual inputs to the VC and from the VC to the PFC are suppressed. (B) ACh is high (high expected uncertainty). Recurrent connections within the PFC and inputs from the PFC to the PPC are suppressed. Visual inputs into the VC and from the VC to the PFC are enhanced. (C) NA is low (low unexpected uncertainty). Gain in the BF and the forgetting factor in the connections from VC to PFC and PFC to PPC are decreased. Sensory inputs to the PPC from VC are suppressed and predictive signals from the PFC to PPC are enhanced. (D) NA is high (high unexpected uncertainty). Gain in the BF and the forgetting factor in the connections from VC to PFC and PFC to PPC are increased. Sensory input to the PPC from VC are enhanced and predictive signals from the PFC to PPC are suppressed.

Figure 4

Simulated behavioral task. Plots showing the neural activity (VC, PFC, PPC), behavior (Head Direction), and neuromodulatory levels ([NA], [ACh]). [NA] and [ACh] track unexpected and expected uncertainty, respectively. For the first 1800 seconds, the Gaussian distribution from which each light position was drawn had μ = 30 and σ = 1 degree (low expected uncertainty). From 1800 to 3600 seconds, a new distribution with μ = 15 and σ = 40 degrees (high expected uncertainty) was chosen, introducing unexpected uncertainty (novelty) into the environment. At 3600 seconds, we set μ = 5 and σ = 10 degrees (medium expected uncertainty). At 5400 seconds, we set μ = 20 and σ = 1 degree (low expected uncertainty). The color map for the first three plots (VC, PFC, PPC) is in gray scale, thus, neurons that are highly active are white, neurons that are inactive are black, and neurons that are in between are gray.

Figure 5

Behavioral task in which the standard deviation is held constant and the mean is changed. In order to see clearer the response of the noradrenergic system to large changes in the environment, we hold the expected standard deviation constant at 10 degrees and change the mean of the distribution. The NA system responds to the changes in unexpected uncertainty with a phasic spike at approximately 1800, 3600, and 5400 seconds.

Figure 6

Behavioral task in which the mean light location is held constant and the standard deviation is increased. In order to see clearer the response of the cholinergic system to expected uncertainty in the environment, we hold the mean of the distribution constant and increase the standard deviation by 10 degrees every 1800 seconds. The ACh system has an increasingly elevated response as the expected uncertainty increases over time.

Figure 7

Behavioral performance in the model. Plots show the mean and standard deviation of correct responses, incorrect responses, and responses where the agent did not make a choice (No Go) over 50 experiments with an intact or simulated lesion to the cholinergic or noradrenergic systems. Asterisks (^*) indicate a statistically significant difference with p-value less than 0.017 (with Bonferroni correction). (A) Low uncertainty. A lesion of BF resulted in significantly less correct responses (p < 1 × 10⁻¹⁶; t-test) and significantly more incorrect responses (p < 1 × 10⁻²⁰; t-test) than the non-lesioned model. (B) High uncertainty. A lesion of BF resulted in significantly less correct responses (p < 1 × 10⁻¹⁴; t-test) and significantly more incorrect responses (p < 1 × 10⁻¹⁵; t-test) than the non-lesioned model. Lesioning the LC, however, lead to a marginally significant decrease in the number of correct responses in the high uncertainty case (p < 0.05) when compared to the non-lesioned model.

Figure 8

Plot showing the pair-wise distance between the VC and PPC and PFC and PPC. Plot shows the pair-wise distance between the activity in VC and in PPC (blue), and between the activity in PFC and in PPC (green). Under conditions of low uncertainty (1–1800 and 5400–7200 seconds), the PPC is driven more by inputs from the PFC rather than VC. In contrast, under conditions of high uncertainty (1801–3600 seconds), the PPC is driven more by inputs from the VC than from the PFC.

Figure 9

Plots showing the neural activity (VC, PFC, PPC), behavior (Head Direction), and neuromodulatory levels ([NA], [ACh]) when the BF has been lesioned. When the BF is lesioned, input to the VC is no longer enhanced in situations of high uncertainty as can be seen by comparing VC activity from 1801 to 3600 seconds in this Figure and in Figure 4. In addition to the weakened input, the overactive LC causes VC information to be gated into the PPC. The combined lack of enhancement of input and activated LC results in an unreliable sensory signal being gated into the PPC in situations of high uncertainty (seen as gaps in activity in the PPC from 1801 to 3600 seconds). This ultimately causes behavioral response (Head Direction) to appear random from 1801 to 3600 seconds.

Figure 10

Plots showing the neural activity (VC, PFC, PPC), behavior (Head Direction), and neuromodulatory levels ([NA], [ACh]) when the LC has been lesioned. Notice that after 5400 seconds the behavioral response (Head Direction) continually moves to previous light position locations. This results from neurons from previous trials persisting in the PFC and PPC, leading to the observed perseverative behavior.

Figure 11

Influence of LC on the ACh level. When the LC is lesioned, the BF no longer correctly tracks expected uncertainty as can be seen in its diminished activity at high when expected uncertainty is high from 1800 to 3600 seconds. Also, after 5400 seconds, the ACh level remains tonically activated, despite the change to low uncertainty. This is due to the over-activation of the prefrontal cortex (see Figure 5) caused by LC's inability to clear previous expectations.

Figure 12

Probability matching via synaptic connections from the PFC to the BF. The figure shows the strength of the synapse from PFC neurons to a BF neuron in high (red) and low (blue) uncertainty environments. When expected uncertainty is low (Gaussian with standard deviation of 1 degree), a small subset of weights from the PFC to the BF become fully depressed (blue line). When expected uncertainty is high (Gaussian with a large standard deviation of 40 degrees), a larger subset of weights from the PFC to the BF become partially to fully depressed (red line).