Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments

Abstract

Neuromodulatory systems, including the noradrenergic, serotonergic, dopaminergic, and cholinergic systems, track environmental signals, such as risks, rewards, novelty, effort, and social cooperation. These systems provide a foundation for cognitive function in higher organisms; attention, emotion, goal-directed behavior, and decision-making derive from the interaction between the neuromodulatory systems and brain areas, such as the amygdala, frontal cortex, hippocampus, and sensory cortices. Given their strong influence on behavior and cognition, these systems also play a key role in disease states and are the primary target of many current treatment strategies. The fact that these systems interact with each other either directly or indirectly, however, makes it difficult to understand how a failure in one or more systems can lead to a particular symptom or pathology. In this review, we explore experimental evidence, as well as focus on computational and theoretical models of neuromodulation. Better understanding of neuromodulatory systems may lead to the development of novel treatment strategies for a number of brain disorders.

Keywords: brain disorders; computational modeling; computational neuroscience; neuromodulation; neuromodulatory systems.

Figure 1

Neuromodulatory system interactions and their role in disease. This figure illustrates how serotonergic (blue), cholinergic (red), noradrenergic (green), and dopaminergic (purple) systems are highly connected to one another as well as cortical and subcortical structures. Their malfunction has been associated with a host of neurological and psychiatric conditions as indicated above each brain region. Gray arrows denote recurrent connections.

Figure 2

The dopaminergic system and its functions. The dopaminergic system, which originates in the VTA and SNc, has been implicated in a wide variety of functions including reward, saliency, uncertainty, and invigoration. These functions are achieved through interactions with the prefrontal cortex, striatum, and hippocampus. It is also reciprocally connected with the three other neuromodulatory systems, further complicating its role in disease states.

Figure 3

Within a column in the PFC, neuromodulators were modeled by changing the strength of inputs from non-preferred directions (D1 receptors) between layer 2/3 neurons in different columns and the strength layer 5 neuronal responses (D2 receptors). This architecture also shows how layer 5 neurons in each column received input from the MD/SC and output to a non-specific inhibitory group and the basal ganglia in order to clear working memory and update other columns, respectively. In this model, Positive and negative symptoms arise from abnormalities in layer 5 outputs to subcortical structures and cognitive symptoms arise from a “leaky” spread of excitation of lateral excitatory inputs within the PFC.

Figure 4

The serotonergic system and its functions. The serotonergic system originates in the Raphe Nucleus of the brainstem and is connected to prefrontal, sensory, limbic, and striatal structures. Serotonin has been associated with a variety a functions including impulsivity, harm aversion, anxious states, punishment, and withdrawal. Experimental and theoretical studies have suggested it has an antagonistic relationship with the dopaminergic system.

Figure 5

Neuromodulation effects in simulation of the Hawk-Dove game. (Left) Architecture of the neural model (Neuromodulatory: Raphe and VTA; TOI-State: Open, Escalate, and Display; and two Action: Escalate and Display). Solid arrows from the TOI-State neurons denote all-to-all connections. The shaded oval and dotted arrows denote plastic connections. Within the Action neurons, the arrowhead denotes an excitatory connection, and the line with the dot at the end denotes an inhibitory connection. (Right) Proportion of actions taken by the Neural agent. Open, Escalate, and Display are states the Neural agent observes, and Escalate (E), Display (D), and Undecided (U) are actions the Neural agent can take. U represents random choice between “E” and “D”. Labels denote the Neural agent's response to the three states. Dove-like strategies are displayed in blue, Hawk-like are displayed in red, and the lack of a strong bias is displayed in yellow. Reproduced from Asher et al. (2010) with permission.

Figure 6

Embodied model of neuromodulation in an open-field test experiment. Experiments were run on an iRobot Create, equipped with a laser range finder and a netbook that contained the neural model and controlled the robot's behavior. (A) Neural model architecture. Sensory events were handled by three binary neurons. These neurons projected to the attentional filter neurons (AchNE) and the dopaminergic and serotonergic neurons (DA and 5-HT). The DA and 5-HT neurons projected to the OFC and mPFC neurons. The most active OFC or mPFC neuron dictated the robot's behavioral state. The AChNE neurons had a modulatory effect on the projection from the DA and 5-HT to OFC and mPFC (see blue ellipse and arrows). OFC and mPFC projected to 5-HT and DA neurons with inhibitory connections. Excitatory and inhibitory connections within and between OFC and mPFC neurons were all-to-all. (B) Wall following behavior. (C) Find home behavior. Finding home consisted of locating the robot's docking station. (D) Open-field behavior. The robot moved toward open spaces in the environment based on laser range finder readings. (E) Explore object. The robot approached narrow objects based on laser range finder readings. Reproduced from Krichmar (2013) with permission.

Figure 7

The noradrenergic system and its functions. The noradrenergic system, which originates in the locus coeruleus, has been implicated in exploration-exploitation trade-off computations and large-scale reorganization of networks in the brain in response to surprise. Locus coeruleus activity is regulated by prefrontal and cingulate cortices and sends its projections throughout the cortex as well as to other neuromodulatory regions such as the basal forebrain.

Figure 8

The cholinergic system and its functions. The cholinergic system originates in the basal forebrain and sends projections to many cortical and subcortical regions. As a result, it has been implicated in a variety of functions including memory, attention, and uncertainty computations. Activity of the basal forebrain is thought to be regulated by prefrontal cortices, as well as other neuromodulatory brain regions.

Figure 9

Neural network model incorporating noradrenergic and cholinergic systems that adapt to uncertainty. (A) The visual input group drives activity in the VC (visual cortex). VC and PFC (prefrontal cortex) provide input to the PPC (posterior parietal cortex). The noradrenergic system, LC (locus coeruleus), enhances the depression of weights (“forgetting”) between VC and PFC, and PFC and PPC [indicated by NA(−)]. The noradrenergic system increases the gain in the BF (basal forebrain) and the input to the PPC from VC [shown by NA(+)] and suppresses input to the PPC from the PFC [shown by the NA(−)]. The cholinergic system, BF, enhances input to VC and PPC [indicated by the ACh(+)] and suppresses recurrent activity in the PFC and input to the PPC from the PFC [indicated by the ACh(−)]. (B) In Experiment 1, the uncertainty level was constant and a surprising stimulus was occasionally presented. NA levels rapidly increased in response to the unexpected stimulus (green), whereas ACh levels rose more gradually. (C) In Experiment 2, surprise was kept constant, but expected uncertainty gradually increased. The figure shows that ACh levels increase as expected uncertainty increases (red). Reproduced from Avery et al. (2012) with permission.