Double-Camera Fusion System for Animal-Position Awareness in Farming Pens

Abstract

In livestock breeding, continuous and objective monitoring of animals is manually unfeasible due to the large scale of breeding and expensive labour. Computer vision technology can generate accurate and real-time individual animal or animal group information from video surveillance. However, the frequent occlusion between animals and changes in appearance features caused by varying lighting conditions makes single-camera systems less attractive. We propose a double-camera system and image registration algorithms to spatially fuse the information from different viewpoints to solve these issues. This paper presents a deformable learning-based registration framework, where the input image pairs are initially linearly pre-registered. Then, an unsupervised convolutional neural network is employed to fit the mapping from one view to another, using a large number of unlabelled samples for training. The learned parameters are then used in a semi-supervised network and fine-tuned with a small number of manually annotated landmarks. The actual pixel displacement error is introduced as a complement to an image similarity measure. The performance of the proposed fine-tuned method is evaluated on real farming datasets and demonstrates significant improvement in lowering the registration errors than commonly used feature-based and intensity-based methods. This approach also reduces the registration time of an unseen image pair to less than 0.5 s. The proposed method provides a high-quality reference processing step for improving subsequent tasks such as multi-object tracking and behaviour recognition of animals for further analysis.

Keywords: convolutional neural network; double-camera; fine-tune; image registration.

Figure 1

Pig pen layout and lens positions.

Figure 2

Image pairs at various stages of pre-processing.

Figure 3

The proposed method consists of two stages. First, train the unsupervised network with image pairs without landmarks to obtain the weights across pig pens and over time; second, import these weights into the semi-supervised network and fine-tune with samples labelled with landmarks.

Figure 4

The architecture of the proposed network.

Figure 5

Examples of two types of landmarks and their correspondences. Red dots indicate background objects and yellow dots indicate the foreground.

Figure 6

Boxplots of TRE scores of each image registration method for aligning the background and the foreground objects. For each method a cluster of three results are grouped: seen pens, unseen pen, and all pens.

Figure 7

Illustrations of non-linear dense registration effects and errors of un-supervised and fine-tuned networks in surveillance of a pen with pigs.

Figure 8

The proposed method retains the appearance features specific to moving images while aligning moving images to fixed images.