Sex differences in the regulation of social and anxiety-related behaviors: insights from vasopressin and oxytocin brain systems

Abstract

To understand how the brain regulates behavior, many variables must be taken into account, with sex as a prominent variable. In this review, we will discuss recent human and rodent studies showing the sex-specific involvement of the neuropeptides vasopressin and oxytocin in social and anxiety-related behaviors. We discuss that sex differences can be evident at pre-pubertal ages as seen in the sex-specific regulation of social recognition, social play, and anxiety by the vasopressin system in juvenile rats. We further discuss that the oxytocin system in humans and rodents alters brain activation, anxiety, and sociosexual motivation in sex-specific ways. Finally, we propose that knowledge of vasopressin and oxytocin mediated sex-specific brain mechanisms can provide essential insights into how these neuropeptide systems contribute to sex-specific vulnerability as well as resilience to perturbations, with subsequent relevance to social and emotional disorders.

Figure 1. Sex differences in the vasopressin (AVP) system in the rat brain and sex-specific regulation of behavior by the lateral septum (LS) AVP system in rats

(A) Sex differences are found in the AVP pathway from bed nucleus of the stria terminalis (BNST) and medial amygdala (MeA) to the lateral septum (LS): 1. Adult male rats have more AVP-immunoreactive (AVP-ir) cells in the posterior BNST than adult female rats, while there are fewer cells and no sex difference in juvenile rats [adapted from 16]. 2. Adult male rats have more AVP-immunoreactive (AVP-ir) cells in the posterodorsal MeA than adult female rats, while there are fewer cells and no sex difference in juvenile rats [adapted from 16]. 3. In the ventral caudal part of the LS, adult and juvenile male rats show denser AVP-ir fibers than females, and adults show denser AVP-ir fibers than juveniles [adapted from 16]; Photomicrographs (scale bar = 100 μm) depict AVP-ir fibers in the ventral caudal part of the LS of a juvenile and adult male and female rat [adapted from 16]. 4. In the dorsolateral LS, juvenile female rats have denser AVP V1a receptor (V1aR) binding than juvenile male rats, and adults show denser V1aR binding than juveniles [adapted from 17]; Autoradiographs show dense V1aR binding in the dorsolateral LS, which includes the dorsal part of the LS (LSd) and the lateral portion of the intermediate part of the LS (LSi) [adapted from 17]. (B) Pharmacological studies in adult rats demonstrate that, despite sex differences in the LS-AVP system, V1aR antagonist impairs and exogenous AVP improves social recognition in both male [17, 19, 20, 22, 23] and female [17] rats. The role of the LS-AVP system is more complicated in juvenile rats: the LS-AVP system regulates social play (V1aR antagonist, exogenous AVP) [27, 28] and social recognition (exogenous AVP) [17] in sex-specific ways. It should be noted that the same dose of the V1aR antagonist d(CH₂)₅[Tyr(Me)²]AVP (10 ng/0.5 μl) and the same dose of AVP (200 pg/0.5μl) were used in [27, 28], suggesting age differences in the role of the LS-AVP system regulating social recognition. Social recognition is reflected by the ability to discriminate between a novel and a familiar same-sex 3-week-old stimulus rat; Social preference is reflected by the preference to investigate a novel conspecific over a novel object; Social novelty preference is reflected by the preference to investigate a novel conspecific over a cage mate. The role of the LS-AVP system in anxiety-related behavior as determined on the elevated plus-maze (EPM) reveals for the most part anxiogenic effects of LS-AVP in adult and juvenile male rats. It should be noted that an increase in anxiety was seen after chronic V1aR antagonist application in the LS [32], while a decrease in anxiety was seen after a single V1aR antagonist application in the LS [31]. (C) The LS has many projections to telencephalon, diencephalon and mesencephalon, with the most notable output to the ventral tegmental area (VTA), nucleus accumbens (NAc), periaqueductal grey (PAG), dorsal raphe nucleus (DRN), hypothalamus (HYP), amygdala (AMY), and thalamus (THA) [12, 35]. It is possible that sex differences in the behavioral effects of LS-AVP manipulations can be, in part, attributed to sex differences in the recruitment of specific LS outputs to mediate behavior. These can include LS-induced changes in mesolimbic reward systems (through e.g., VTA and Nac), neuroendocrine and autonomic projections (through e.g., HYP, PAG, and indirectly via AMY), alterations in monoamine functioning (serotonin through DRN connection and norepinephrine through LC connection, not shown). aca, anterior part of the anterior commissure; cc, corpus collosum; Cpu, caudate putamen; EPM, elevated plus-maze; F, female; FFA, fiber fractional area; LSd, dorsal part of the lateral septum; LSi, intermediate part of the lateral septum; LSv, ventral part of the lateral septum; M, male; *p < 0.05.

Figure 2. Sex differences in the oxytocin (OXT) system in the rat brain and sex-specific effects on brain activation and behavior by the OXT system

(A) Analysis of the rat OXT system reveals sex differences in some, but not all OXT parameters: 1. No sex differences are found in OXT mRNA expression in the paraventricular nucleus (PVN) and supraoptic nucleus (SON) of the hypothalamus in adult rats [adapted from 41]. The autoradiogram depicts OXT mRNA expression (black) in the left PVN and left SON in a 16-μm coronal brain section of an adult male rat [adapted from 41]. 2. No sex differences are found in OXT-immunoreactive (OXT-ir) fiber density (expressed as fiber fractional area or FFA) in the bed nucleus of the stria terminalis (BNST) of juvenile and adult rats [adapted from 16]. Photomicrographs depict OXT-ir fibers in the BNST of a male and female adult rat [adapted from 16]; Scale bar indicates 100 μm. 3. Juvenile male rats have more Fos-positive OXT-ir neurons in the SON than females [38]; Fos is an immediate early gene used as marker for neuronal activation. 4. Adult male rats show higher extracellular OXT release (calculated as percentage of baseline OXT release) in the posterior BNST compared to females during both trials of the social discrimination tests in which the rats were exposed to an unfamiliar sex-matched juvenile rat during trial 1 (T1) and the same previously encountered unfamiliar juvenile rat (now familiar) along with a second unfamiliar sex-matched juvenile rat during trial 2 (T2) [adapted from 39]. 5. Juvenile and adult male rats show denser OXT receptor binding in the posterior BNST compared to females [adapted from 18]. Autoradiograms show representative OXT receptor binding in the posterior BNST of an adult male and adult female rat [adapted from 18]. (B) Intranasal application of OXT in humans and intracerebroventricular administration of OXT in rats induce sex-specific blood-oxygen-level dependent (BOLD) activation of the amygdala, nucleus accumbens, and insula [–50]. In the human studies, men and women were exposed to fearful and/or threatening images or scenes (Fear/threat) or were exposed to an interactive social game (the Prisoner’s Dilemma Game) to examine cooperative interactions (Cooperation). Images depict coronal sections of the human brain (source: https://msu.edu/user/brains/brains/human/) and rat brain [source: 64]. Colors in the coronal brain section depict the amygdala (blue), nucleus accumbens (green), and insula (red). (C) Studies in mice show that the OXT receptor (OTR) in the medial prefrontal cortex (mPFC) regulates anxiety-related behavior and sociosexual motivation in sex-specific ways [51, 52]. In contrast, the mPFC-OTR in rats regulates anxiety-related behavior similiary in males and females [53]. Moreover, the mPFC-OTR is involved in maternal care (pup retrieval), maternal aggression, and anxiety in lactating rats [55]. (D) Studies in mice have shown that in vitro optogenetic stimulation of OTR-expressing interneurons induces a stronger inhibitory postsynaptic current (IPSC) in layer 2/3 pyramidal neurons (important for intra-mPFC connectivity) in males compared to females and a stronger IPSC in layer 5 pyramidal neurons (important for output to subcortical regions) in females compared to males [52]. * p<0.05 versus females.

Figure 3. Unraveling the neuronal mechanisms that makes one sex vulnerable and the other sex resilient may help to improve treatment options for social and emotional disorders

AVP and OXT regulate social behaviors in sex-specific ways, suggesting that perturbations of AVP/OXT systems have different behavioral consequences for males compared to females and could cause one sex to be more vulnerable and the other sex to be more resilient to the development of a specific social or emotional disorder. Studying the mechanisms underlying the sex-specific behavioral effects of AVP and OXT systems will not only provide a more complete understanding of the function of the brain for behavior, but also has the potential to provide insights into sex-specific vulnerability and resilience to these neuropsychiatric disorders. PTSD, posttraumatic disorder; BPD, borderline personality disorder.