The MacqD deep-learning-based model for automatic detection of socially housed laboratory macaques

Abstract

Despite advancements in video-based behaviour analysis and detection models for various species, existing methods are suboptimal to detect macaques in complex laboratory environments. To address this gap, we present MacqD, a modified Mask R-CNN model incorporating a SWIN transformer backbone for enhanced attention-based feature extraction. MacqD robustly detects macaques in their home-cage under challenging scenarios, including occlusions, glass reflections, and overexposure to light. To evaluate MacqD and compare its performance against pre-existing macaque detection models, we collected and analysed video frames from 20 caged rhesus macaques at Newcastle University, UK. Our results demonstrate MacqD's superiority, achieving a median F1-score of 99% for frames with a single macaque in the focal cage (surpassing the next-best model by 21%) and 90% for frames with two macaques. Generalisation tests on frames from a different set of macaques from the same animal facility yielded median F1-scores of 95% for frames with a single macaque (surpassing the next-best model by 15%) and 81% for frames with two macaques (surpassing the alternative approach by 39% ). Finally, MacqD was applied to videos of paired macaques from another facility and resulted in F1-score of 90%, reflecting its strong generalisation capacity. This study highlights MacqD's effectiveness in accurately detecting macaques across diverse settings.

Keywords: Animal behaviour; Automatic detection; Deep learning; Macaques; Non-human primate; Pair-housed.

Fig. 1

Example video frames used in this study. (a) Single macaque, partially hidden, with light overexposure; (b) Single macaque with cage railing occlusion; and (c) Pair of macaques in the focal cage, with partial overlap of the two individuals, partial occlusion from cage enrichment and one macaque from a neighbouring cage appearing in the background.

Fig. 2

Overview of the Mask R-CNN framework for macaque detection.

Fig. 3

Overview of the background elimination pipeline.

Fig. 4

Overview of experiments 1 and 2 illustrating how the models compared in this study differed in terms of training and testing datasets. ‘Same’ and ‘Different’ correspond to the datasets described in Table 1, with the labels referring to the fact that the model was tested with videos of individual macaques ‘seen’ during the training phase (‘same’) or not (‘different’) (see section “Data description” for more details).

Fig. 5

Tracking algorithm pipeline.

Fig. 6

Model performance in Experiment 1, where the test datasets contain only a single macaque in the focal cage. (a–c) represent precision, recall, and F1 scores, respectively, evaluated on datasets featuring the same macaques (but different videos) as the training dataset (‘Same’ dataset). (d–f) represent precision, recall, and F1 scores, respectively, evaluated on datasets featuring macaques not present in the training dataset (‘Different’ dataset). Markers represent results for individual macaques, with lines connecting the markers across models to illustrate performance variations. Wilcoxon test: *, **.

Fig. 7

Model performance in Experiment 2, where the test datasets contain paired macaques in the focal cage. (a–c) represent precision, recall, and F1 scores, respectively, evaluated on datasets featuring the same macaques (but different videos) as the training dataset (‘Same’ dataset). (d–f) represent precision, recall, and F1 scores, respectively, evaluated on datasets featuring macaques not included in the training dataset (‘Different’ dataset). Markers represent results for individual macaques, with lines connecting the markers across models to illustrate performance variations. Wilcoxon test: *, **.

Fig. 8

Comparison of ground truth versus predicted segmentations/bounding boxes for frames from unseen animals. The first two columns show MacqD - Macaque Single’s predictions on frames featuring single animals not present in the training dataset (Experiment 1, ‘Different’ dataset). The third and fourth columns show MacqD - Macaque Combine’s predictions on frames featuring pairs of macaques not present in the training dataset (Experiment 2, ‘Different’ dataset).

Fig. 9

Comparison of segmentations predicted by MacqD – Macaque Curriculum and ground-truth bounding boxes on a frame from the ISC dataset.