Instance segmentation and automated pig posture recognition for smart health management

Abstract

Changes in posture and movement during the growing period can often indicate abnormal development or health in pigs, making it possible to monitor and detect early morphological symptoms and health risks, potentially helping to limit the spread of infections. Large-scale pig farming requires extensive visual monitoring by workers, which is time-consuming and laborious. However, a potential solution is computer vision-based monitoring of posture and movement. The objective of this study was to recognize and detect pig posture using a masked-based instance segmentation for automated pig monitoring in a closed pig farm environment. Two automatic video acquisition systems were installed from the top and side views. RGB images were extracted from the RGB video files and used for annotation work. Manual annotation of 600 images was used to prepare a training dataset, including the four postures: standing, sitting, lying, and eating from the food bin. An instance segmentation framework was employed to recognize and detect pig posture. A region proposal network was used in the Mask R-CNN-generated candidate boxes and the features from these boxes were extracted using RoIPool, followed by classification and bounding-box regression. The model effectively identified standard postures, achieving a mean average precision of 0.937 for piglets and 0.935 for adults. The proposed model showed strong potential for real-time posture monitoring and early welfare issue detection in pigs, aiding in the optimization of farm management practices. Additionally, the study explored body weight estimation using 2D image pixel areas, which showed a high correlation with actual weight, although limitations in capturing 3D volume could affect precision. Future work should integrate 3D imaging or depth sensors and expand the use of the model across diverse farm conditions to enhance real-world applicability.

Keywords: Computer vision; Pig activity; Pig identification; Pig posture; Segmentation; Smart agriculture.

Fig. 1.. The pig farm site and the pig room used for this experimental setting.

(A) The overall pig farm site, (B) pig room where the study took place, (C) the piglets within the pig pen, and (D) adult pigs housed in the similar pens.

Fig. 2.. Data acquisition setup used in the pig farm, showing the positions of the microcontroller and camera from both top and side views.

The setup was designed to capture pig posture data effectively for subsequent analysis.

Fig. 3.. A demonstration of the annotation process for pig posture detection, conducted using the open-source online platform MakeSense.ai, and highlighting the steps involved in labeling and preparing the data for training the detection model.

Fig. 4.. Visual examples of the four posture classes observed in piglets and pigs.

The postures (sitting, lying, eating, and standing) show variations between piglets and pigs, aiding in the understanding of how these postures are monitored for health assessments.

Fig. 5.. Illustration of the improved Mask–RCNN architecture applied in this study for pig posture detection.

It included key components such as the ResNeXt–101 backbone and feature pyramid network (FPN), and regional proposal network (RPN) algorithm, showing how input images are processed to generate class, bounding box, and mask outputs for accurate posture detection.

Fig. 6.. Unit Structure of (A) ResNet-101 and (B) ResNeXt-101 architectures.

The components include convolutional layers (Conv), batch normalization layers (BN), and ReLU activation functions. In (B), the ResNeXt-101 architecture is shown with grouped convolutions, indicated by “F/32” for the number of feature maps, which is designed to improve feature learning and computational efficiency.

Fig. 7.. Outputs of feature extraction for various pig postures using the improved algorithm.

Each row represents different postures of piglets and pigs (sitting, lying, eating, and standing), with the extracted features highlighted in the corresponding columns. The size of each image is denoted as H × W = 480 × 800, illustrating the segmentation results for each posture class.

Fig. 8.. The process of pig body weight estimation through segmented pixel numbers.

(A) An original image, (B) the annotated image with different colors indicating detected pigs, (C) a masked image showing detected areas, (D) ground truth segmentation derived from the annotated image, and (e) the segmented results obtained from the masked image. The segmented areas were used to estimate body weight by counting the pixel numbers corresponding to each pig.

Fig. 9.. Performance evaluation of the improved Mask R–CNN model for pig posture detection across 100 epochs.

(A) training and validation loss curves, indicating the decrease in loss during model training, and (B) training and validation accuracy curves, highlighting the increase in accuracy over time. These results demonstrated the effectiveness of the model in accurately detecting pig postures.

Fig. 10.. The mAP curves for the pig posture detection model across 100 epochs.

The blue line represents the mAP@50, and the brown line shows the mAP@50:95, illustrating the precision in detecting pig postures at different intersection-over-union thresholds. Both curves show improvement as training progresses, indicating an increasing accuracy in posture detection.

Fig. 11.. Output results of piglet posture detection and segmentation in test images using the proposed mask R–CNN model.

The postures were labeled using different colors in annotated images (upper row), while in detected images (lower row), the postures wer highlighted with bounding boxes and confidence scores, demonstrating the ability of the model to accurately identify and segment piglet postures in test images.

Fig. 12.. Inaccurate piglet posture detection and segmentation in test images using the proposed Mask R–CNN model (marked with blue rectangles).

(A) Misdetection, where the model incorrectly identifies a posture; (B) overlap detection, where two postures are mistakenly detected together; and (C) a case where the posture is not detected, despite being present in the image. These examples illustrate areas where the accuracy of the model could be improved.

Fig. 13.. Results of pig posture detection and segmentation using the proposed Mask R–CNN model in test images.

In the annotated images (upper row), different postures are marked with various colors, whereas in the detected images (lower row), the postures are segmented and assigned confidence scores with bounding boxes, demonstrating the effectiveness of the model in accurately identifying and segmenting different pig postures.

Fig. 14.. Inaccurate pig posture detection and segmentation in test images using the proposed Mask R–CNN model (marked as blue rectangles).

(A) certain postures are not detected, (B) a posture is missed, and (C) another posture is incorrectly detected. These instances highlight the limitations of the model in some cases, despite its overall accuracy.

Fig. 15.. Variability in pig behaviors over 24 hours.

(A) A comparison of eating and standing behaviors, and (B) a comparison of sitting and lying behaviors. The plots show the percentage of time spent in each posture across the day, highlighting trends and patterns in pig behavior.

Fig. 16.. The relationship between actual pig body weight and pixel numbers derived from the segmented pig body area.

(A) The correlation between body weight and pixel numbers for piglets and pigs, and (B) temporal changes in actual and predicted pig body weight over the experimental period. The data show trends in weight estimation based on the pixel counts during the monitoring days.