ABNet: AI-Empowered Abnormal Action Recognition Method for Laboratory Mouse Behavior

Abstract

The automatic recognition and quantitative analysis of abnormal behavior in mice play a crucial role in behavioral observation experiments in neuroscience, pharmacology, and toxicology. Due to the challenging definition of abnormal behavior and difficulty in collecting training samples, directly applying behavior recognition methods to identify abnormal behavior is often infeasible. This paper proposes ABNet, an AI-empowered abnormal action recognition approach for mice. ABNet utilizes an enhanced Spatio-Temporal Graph Convolutional Network (ST-GCN) as an encoder; ST-GCN combines graph convolution and temporal convolution to efficiently capture and analyze spatio-temporal dynamic features in graph-structured data, making it suitable for complex tasks such as action recognition and traffic prediction. ABNet trains the encoding network with normal behavior samples, then employs unsupervised clustering to identify abnormal behavior in mice. Compared to the original ST-GCN network, the method significantly enhances the capabilities of feature extraction and encoding. We conduct comprehensive experiments on the Kinetics-Skeleton dataset and the mouse behavior dataset to evaluate and validate the performance of ABNet in behavior recognition and abnormal motion detection. In the behavior recognition experiments conducted on the Kinetics-Skeleton dataset, ABNet achieves an accuracy of 32.7% for the top one and 55.2% for the top five. Moreover, in the abnormal behavior analysis experiments conducted on the mouse behavior dataset, ABNet achieves an average accuracy of 83.1%.

Keywords: action recognition; computer vision; mice; mouse behavior; semi-supervised learning.

Figure 1

ABNet overall process.

Figure 2

Skeleton points of a mouse.

Figure 3

Comparison of keypoint detection between DeepLabCut and YOLOv9. The first line is the detection results of the DeepLabCut pose estimation algorithm, and the second line is the detection results of the YOLOv9 object detection algorithm.

Figure 4

Overview of the DeepLabCut network architecture.

Figure 5

ST-GCN network structure.

Figure 6

Diagram of GCN principles.

Figure 7

Diagram of TCN principles.

Figure 8

Overview of the SE module structure.

Figure 9

Overview of the improved ST-GCN network.

Figure 10

Overview of multi-branch TCN.

Figure 11

Visualization of clustered feature information, with black outliers identified as abnormal behavior. The red dots represent movement, the blue dots represent turning, the yellow dots represent standing, the gray dots represent head turning, and the black dots represent abnormal behavior.