CalliCog: an open-source cognitive neuroscience toolkit for freely behaving nonhuman primates

Abstract

Motivation: Cognitive neuroscience research involving nonhuman primates (NHPs) has traditionally been confined to a few highly specialized laboratories equipped with advanced infrastructure, expert knowledge, and specialized resources for housing and testing these animals. The common marmoset (Callithrix jacchus), a small NHP species, has gained popularity in cognitive research due to its ability to address some of these challenges. However, behavioral studies in marmosets remain labor-intensive and restricted mainly to experts in the field, making them less accessible to the broader scientific community. To address these barriers, we introduce an open and accessible platform designed for automated cognitive experiments in home cage settings with marmosets. This system supports the integration of cognitive behavioral analysis with wireless neural recordings, is cost-effective, and requires minimal technical expertise to build and operate.

Summary: Nonhuman primates (NHPs) are pivotal for unlocking the complexities of human cognition, yet traditional cognitive studies remain constrained to specialized laboratories. To revolutionize this paradigm, we present CalliCog: an open-source, scalable in-cage platform tailored for freely behaving experiments in small primate species such as the common marmoset (Callithrix jacchus). CalliCog includes modular operant chambers that operate autonomously and integrate seamlessly with home cages, eliminating human intervention. Our results showcase the power of CalliCog to train experimentally naïve marmosets in touchscreen-based cognitive tasks. Remarkably, across two independent facilities, marmosets achieved touchscreen proficiency within two weeks and successfully completed tasks probing behavioral flexibility and working memory. Moreover, CalliCog enabled precise synchronization of behavioral data with electrocorticography (ECoG) recordings from freely moving animals, opening new frontiers for neurobehavioral research. By making CalliCog openly accessible, we aim to democratize cognitive experimentation with small NHPs, narrowing the translational gap between preclinical models and human cognition.

Keywords: Automation; behavior; behavioral flexibility; cognition; electrocorticography; marmoset; prefrontal cortex; working memory.

Figure 1.. High-throughput, automated cognitive experiments in a marmoset housing facility using CalliCog.

Individual home cages are fitted with operant chambers (inset image), which are used to train and test behaviorally naïve animals in touchscreen-based cognitive tasks using positive reinforcement. The functions of each operant chamber are controlled by an internal agent PC, which receives trial instructions over a local network from an executive PC housed outside of the housing area. The executive PC receives behavioral data from the agent PC and queries performance against defined parameters to determine automated task progression. A single executive PC can simultaneously administer multiple operant chambers in a housing facility, facilitating a flexible and scalable approach to home cage testing. In addition, devices for neural recording can be integrated with the local network to study the neural correlates of cognitive behavior. In this example, an off-the-shelf system for wireless telemetry recording (indicated by green arrows) is used to acquire ECoG via wireless receivers while animals engage in touchscreen-based tasks. *Agent PCs are synchronized to a clock set by the telemetry PC using a conventional network time protocol, allowing for synchronous behavioral and neural data timing.

Figure 2.. Workflow for automated progression during behavioral training or testing.

Once a protocol has been designed and initiated by the user at the executive PC, behavioral experiments run autonomously by evaluating animal performance on a trial-by-trial basis. The executive PC sends trial instructions to an agent PC using the JSON text format, which the agent PC parses for trial execution. Trial data, including all timestamps of display events and touch responses, are then returned to the executive PC and logged via the SQL programming language to a local relational database (PostgreSQL). The executive PC then evaluates the logged trial data against predefined ‘progression criteria’ to determine whether to continue the current task or progress to the next in sequence. Responding to animal performance at single-trial resolution allows the system to adapt tasks based on cognitive behaviors that can change on a single moment (e.g. learning or decision-making).

Figure 3.. Experimental pipeline and animal performance in automated touchscreen training.

A. Summary of the unsupervised experimental pipeline, including all stages of training and testing, used in the current study. Each experiment incorporated multiple phases through which animals progressed after achieving predefined progression criteria. Three types of progression criteria were used: ‘session-based’, where animals performed consecutive sessions of trials to a predefined rate of success; ‘rolling average’, where animals performed to a predefined success rate over a rolling window of trials; and ‘target-based’, where animals completed absolute numbers of trials before progression, regardless of successful or failed outcome. B. The touchscreen training protocol included 7 individual phases in which animals were gradually trained to respond to small (250 × 250 px) stimuli regardless of color and spatial position. C, D. Summarized performance of the experimental cohort over all 7 phases of touchscreen training. The data is presented to show the variability between animals that were housed in different animal facilities. On average, animals completed touchscreen training in 44.1 sessions or 14 days. E. Variability between the performance of individual animals on phases 2–7 of the touchscreen training protocol. Performance across individual phases did not vary significantly (p = 0.97; Kruskal-Wallis test). Note that animals require a minimum number of 3 sessions to complete each phase.

Figure 4.. Animal performance in behavioral flexibility and working memory tests using CalliCog.

A. Trial representation of the novel discrimination and reversal learning tasks. Animals interact with a cue to initiate a trial. Then, they must discriminate a rewarded stimulus from a non-rewarded stimulus from a pair of novel stimuli, pseudorandomly presented in left and right positions. Per stimulus pair, animals perform a novel discrimination stage to proficiently learn the stimulus-reward contingencies, which are then instantly reversed in the reversal learning stage after a progression criterion is reached. Behavioral flexibility is evaluated based on performance in adapting responses during reversal learning. B. Performance on the 3 stimulus pairs used for novel discrimination training. C. Performance on the 10 stimulus pairs used for novel discrimination testing. Stimulus pair 9 significantly contributed to a main effect of stimulus pair on performance (Kruskal-Wallis test with Dunn’s test, p = 0.015), so was omitted from subsequent analysis. D. Averaged learning curves of individual animals across all 10 reversal learning experiments. The data is smoothed for visualization purposes using a 2^nd order polynomial fit to a rolling window of 100 neighbors. E. Individual performance in reversal learning experiments between animals, as characterized by the number of errors committed before reaching criterion. The mean performance did not differ significantly between animals (p = 0.087; Kruskal-Wallis test). F. The proportion of trials animals performed across 3 phases of reversal learning, as calculated from learning curves of cumulative performance using a method for learning curve analysis. Animals performed significantly more trials in the achieving phase than in the perseverative phase (Kruskal-Wallis test with Dunn’s test; p = 0.015). The learning phase, in which animals adapted their responses to the new contingencies more consistently, was the most variable. Data points represent the mean number of trials per animal across all 9 reversal experiments. G. The probability of animals using win-stay or lose-shift strategies during the learning phase of reversal learning represents the likelihood of correctly adapting responses based on positive and negative feedback, respectively. No significant differences were observed between either strategy (Mann-Whitney U test; p = 0.535). H. Trial representation of the delayed match-to-sample task (DMTS) used to assess working memory maintenance. Animals are presented with a sample stimulus to be recalled, pseudorandomly selected from a yellow circle, red diamond, or blue star. After interacting with the sample twice, a variable delay period of between 0.5 – 12 s is presented, and animals must then choose the sample in the presence of a distractor to perform the task successfully. The relationship between successful performance and the length of the delay period indicates the capacity for maintaining working memory over time. I. The average working memory decay curve generated from the averaged performance data of animals (n = 3) performing the DMTS. The data is fit to a simple exponential decay function, and the chance threshold is the performance value at which the percentage of success is significantly greater than the chance performance of 50%. J, K. Derived metrics of zero-delay performance and performance half-life from the working memory decay curves of individual animals, which represent the baseline ability to perform the task and the rate of working memory decay, respectively. * p < 0.05

Figure 5.. Task-related neural recording in CalliCog.

A. Workflow for task-related ECoG analysis in freely behaving marmosets. Animals (n = 2) were implanted with 2-channel ECoG electrodes over the bilateral prefrontal cortex, transmitting live data wirelessly during performance in the DMTS. Five timestamps were recorded from all DMTS trials to define 4 epochs from ECoG data relevant to working memory: encoding, maintenance, retrieval, and outcome. These epochs could then be segmented from the recorded data and analyzed independently. B-D. Averaged potentials of filtered, preprocessed ECoG recorded from animals M2 and M3 while performing successful trials at 3 exemplary delay periods of 1, 2, and 4 s (n = 70 trials per delay condition). The dotted lines represent the onset and the end of the maintenance epoch. Averaged activity was stereotyped between animals and delay lengths, and evoked events were aligned to the onset of the delay period and following the end of the delay during working memory retrieval. E. The averaged power spectra from individual epochs of successful trials from animal M2. Power spectral densities were estimated using a multitaper method, and the data was and log-transformed for visualization purposes. Frequency values between 50 – 70 Hz were included in notch filtering and this excluded from the analysis. F. Power quantifications from the power spectra of each epoch at various frequency bands of interest. ECoG power underwent re-weighting between epochs of the DMTS, with all frequency bands differentially recruited in a minimum of one epoch (p < 0.05, Friedman test with Dunn’s test for multiple comparisons). Power is expressed as the absolute power of the non-transformed spectrum within each frequency band relative to the absolute power of the entire spectrum. * p < 0.05, ** p < 0.01, *** p < 0.001, **** p < 0.0001