Tracking conditioned fear in pair-housed mice using deep learning and real-time cue delivery

Abstract

Post-traumatic stress disorder (PTSD) is a complex and prevalent neuropsychiatric condition that arises in response to exposure to a traumatic event. A common diagnostic criterion for PTSD includes heightened physiological reactivity to trauma-related sensory cues, in safe or familiar environments. Understanding complex PTSD criteria requires new pre-clinical paradigms and technologies that integrate sensory physiology (e.g., auditory, visual, olfactory) with behavior. Here we present a novel Pavlovian-based paradigm using an open-source software plus deep learning-based pose estimation to investigate the effects of a recurrent conditioned stimulus (CS) on fear behaviors in pair-housed mice within the home cage. Simultaneous home cage video recording and analysis of CS-evoked freezing behaviors were performed using a deep learning model, with consideration for light-dark circadian cycles. Fear-conditioned dyad mice exhibited high CS-evoked freezing, with evidence of extinction learning (characterized by low freezing) during the mid-phase of the 2-week paradigm. Females exhibited reduced CS-evoked home cage freezing compared to males with circadian differences between the light (low freezing) and dark (high freezing) periods. Following the 2-week paradigm, fear-conditioned mice, compared to controls, exhibited heightened context-dependent freezing, while males but not females showed heightened startle reactivity. Taken together, these results demonstrate a novel software application for examining conditioned defensive and fear behaviors over time in mouse dyads within an ethologically relevant environment. Future applications could be used for more integrative analysis and understanding of neural circuits and heightened sensory threat reactivity, potentially improving the understanding and treatment of PTSD.

Keywords: (six): PTSD; Freezing; Home cage; Pavlovian fear conditioning.

Fig. 1

Protocol and experimental design. A. Integration of Adaptive Cue Delivery (ACD) with behavioral monitoring (1) ACD delivers conditioned auditory cues at random intervals over 24 h (2) Cues play through individual speakers in each cage of pair-housed mice (3) Cages are enclosed with sound-attenuating panels to standardize volume (4) Top-down cameras record freezing behavior continuously across light/dark cycles. B. Fear conditioning protocol and 14-day home-cage cue exposure. Conditioned auditory cues were delivered intermittently in the home cage over 14 days following fear conditioning. C. Machine learning analysis and behavioral phenotyping.

Fig. 2

Assessment of Circadian Freezing Behavior Following Fear Conditioning in the Home Cage of pair-housed Mice A. Schematic of fear conditioning protocol (20 CS/US pairings). B–C. Fear acquisition measured by percent freezing in male (n = 16; B) and female (n = 10; C) mice. D. Male mice show significantly greater freezing during conditioning compared to females (group effect: F(1,22) = 8.552, p < 0.01). E. Diagram of home-cage video monitoring system. F, H. Manual scoring of CS-evoked freezing in fear-conditioned (FC) vs. non-fear-conditioned (Non-FC) group-housed mice across 14 days (time: F(13,315) = 20.25, p < 0.0001; group: F(1,30) = 548.1, p < 0.0001; interaction: F(13,315) = 12.87, p < 0.0001). G, I. CS-evoked freezing in male (G) and female (I) mice across light/dark cycles (males: F(1,30) = 8.423, p = 0.0069; females: F(1,18) = 10.76, p = 0.0042). J. Sex differences in circadian CS-evoked freezing behavior (day–night × sex interaction: F(13,276) = 3.263, p = 0.0001).

Fig. 3

Machine learning-based tracking and analysis of CS-evoked freezing in pair-housed mice using DeepLabCut™ and SimBA™. A. Overview of experimental setup and automated tracking pipeline. B. Validation workflow for deep learning-based percent freezing detection. C–D. Automated analysis of thresholded freezing behavior (binned in 10-s intervals; see Video 2), converted to CS-evoked percent freezing in fear-conditioned (FC) and non-fear-conditioned (Non-FC) pair-housed mice (n = 4 cages). E–H. Comparison of automated vs. manual scoring across 14 days in male Non-FC (E), male FC (F), female Non-FC (G), and female FC (H) groups. I. Violin plot showing the distribution of total CS-evoked percent freezing across all groups (n = 8–12); treatment effect: F(7,104) = 37.24, p < 0.0001.

Fig. 4

Sex-specific behavioral outcomes following fear conditioning (FC) and pseudo-randomized intermittent CS exposure in the home cage. (A) Experimental approach for assessing fear expression in a novel context (Day 15). (B) Percent (%) freezing across Pre-CS, CS1, and CS2 periods. FC groups (gray/blue) exhibited robust freezing, while Non-FC groups (light gray/light blue) did not. (C) FC-induced freezing in a novel context (Day 15) was greater in males than females (treatment effect: F(3,40) = 7.751, p = 0.0003). (D) FC males, but not females, show increased startle reactivity to a 120 dB white noise burst after 14 days of pseudorandomized CS exposure (treatment effect: F(3,40) = 17.03, p < 0.0001). (E) Startle leader trial traces for males (top) and females (bottom), showing trial-by-trial modulation by FC. (F) Schematic of fear-potentiated startle testing (Day 18), involving auditory CS presentations paired with startle stimuli. (G) FC males also show potentiated startle responses to the CS, not observed in females (treatment effect: F(3,38) = 3.632, p = 0.0213). (H) Startle traces from cue-potentiated startle test trials, separated by sex and condition. (I) Elevated Plus Maze (EPM) schematic for Day 19 avoidance/anxiety-like behavior testing. (J–K) % time spent in open arms (J) and number of open arm entries (K) were not significantly affected by FC in either sex.

figs1