Hypothalamic Interactions with Large-Scale Neural Circuits Underlying Reinforcement Learning and Motivated Behavior

Abstract

Biological agents adapt behavior to support the survival needs of the individual and the species. In this review we outline the anatomical, physiological, and computational processes that support reinforcement learning (RL). We describe two circuits in the primate brain that are linked to specific aspects of learning and goal-directed behavior. The ventral circuit, that includes the amygdala, ventral medial prefrontal cortex, and ventral striatum, has substantial connectivity with the hypothalamus. The dorsal circuit, that includes inferior parietal cortex, dorsal lateral prefrontal cortex, and the dorsal striatum, has minimal connectivity with the hypothalamus. The hypothalamic connectivity suggests distinct roles for these circuits. We propose that the ventral circuit defines behavioral goals, and the dorsal circuit orchestrates behavior to achieve those goals.

Keywords: amygdala; basal ganglia; devaluation; frontal-striatal circuits; hypothalamus; motivation; prefrontal cortex; reinforcement learning; striatum.

Fig. 1.

Dorsal and ventral circuits in the primate brain for goal-directed behavior. A. Overview of the proposed dorsal circuit, displayed on schematics of the macaque brain (lateral view, top right), medial dorsal thalamus (top left), striatum (middle) and globus pallidus (bottom left). Dorsal circuit connecting inferior parietal cortex (area 7a) with dorsal lateral prefrontal cortex (dlPFC, area 46). The cortical circuits have overlapping projections in the dorsal striatum and lateral portion of the medial dorsal thalamus (MDl). The striatal circuit also connects through the globus pallidus internal segment (GPi). B. Overview of the proposal ventral circuit (as in A, only with a ventral view of the brain in top right). The ventral circuit (red lines) connects the basolateral amygdala (BLA) and adjacent medial temporal lobe cortex including entorhinal (EC) and perirhinal (PrC) cortex to the ventral medial prefrontal cortex (vmPFC). The vmPFC and BLA/EC/PrC have convergent projections in the ventral striatum (VS) and medial MD thalamus. The VS connects through the ventral-pallidum (VP) back to the medial portion of the medial dorsal thalamic nucleus (MDm). The ventral circuit is generally consistent with the classical definition of the limbic system. However, considering the parallel organization of ventral and dorsal circuits may help understand their function. C. Schematic illustration of the dorsal, ventral and amygdala circuits. Many connections are not shown, for clarity. Note the substantial anatomical connectivity between the amygdala and the ventral circuitry. The ventral circuitry has substantial projections to the hypothalamus (Hypo), whereas the dorsal circuit has minimal connections to the hypothalamus. While the amygdala projects to the ventral circuit, it also has a parallel cortical-basal ganglia-thalamo-cortical architecture [30]. In this conception, the basolateral amygdala (BLA) is the (allo)cortical node. The BLA projects to the central nucleus (CeN) of the amygdala, which is part of the striatum. The CeN then projects to the bed nucleus of the stria terminalis (BNST; equivalent to the VP). The BNST projects to the paraventricular thalamus (PVT), which projects back to the amygdala, closing the loop [98]. There is also a hippocampal circuit, that largely parallels the amygdala circuit, which we have not shown for simplicity (see Box 3 for some comments on the hippocampus). This circuit projects through the lateral septum (LS) and medial septum, back through the reuniens (Re) nucleus of the thalamus.

Fig. 2.

Connectivity of ventral circuits with the hypothalamus. Colored lines match colors of each circuit from Fig. 1C. The vmPFC, BNST, VP and VS all send inputs to the hypothalamus. LH: lateral hypothalamus. Arc: arcuate nucleus of the hypothalamus. NTS: nucleus of the solitary tract; PB: parabrachial nucleus; see figure 1 for additional abbreviations. Note that we are illustrating connections from the surface of the medial temporal lobe. The basolateral amygdala, EC and PrC have similar connections.