Identifying prey capture events of a free-ranging marine predator using bio-logger data and deep learning

Abstract

Marine predators are integral to the functioning of marine ecosystems, and their consumption requirements should be integrated into ecosystem-based management policies. However, estimating prey consumption in diving marine predators requires innovative methods as predator-prey interactions are rarely observable. We developed a novel method, validated by animal-borne video, that uses tri-axial acceleration and depth data to quantify prey capture rates in chinstrap penguins (Pygoscelis antarctica). These penguins are important consumers of Antarctic krill (Euphausia superba), a commercially harvested crustacean central to the Southern Ocean food web. We collected a large data set (n = 41 individuals) comprising overlapping video, accelerometer and depth data from foraging penguins. Prey captures were manually identified in videos, and those observations were used in supervised training of two deep learning neural networks (convolutional neural network (CNN) and V-Net). Although the CNN and V-Net architectures and input data pipelines differed, both trained models were able to predict prey captures from new acceleration and depth data (linear regression slope of predictions against video-observed prey captures = 1.13; R ² ≈ 0.86). Our results illustrate that deep learning algorithms offer a means to process the large quantities of data generated by contemporary bio-logging sensors to robustly estimate prey capture events in diving marine predators.

Keywords: accelerometer; animal behaviour classification; animal-borne video; chinstrap penguin; foraging ecology; machine learning.

Figure 1.

Workflow from data acquisition to model output. (a) Bio-logging data (tri-axial acceleration, depth and video) are time synchronized and labelled where PCE are observed in the video. (b) The input data used in the models are the three accelerometer axes, which are (c) balanced through dive thresholds for CNN and using a global generator (fig. 3 in [37]) with random sampling windows for V-Net. (d) The data are fed into the CNN model in windows of eight data points with 50% overlap, and windows of 128 data points with no overlap for V-Net. (e) The trained models are used to predict PCE for 15 individuals (not used during training) and predicted PCE are regressed against the labelled PCE for each dive to assess model performance.

Figure 2.

A sequence of frames (a–d) lasting 0.16 s shows a chinstrap penguin catching an individual krill from a small swarm.

Figure 3.

Example of labelled and predicted PCE from two chinstrap penguins, showing high-density foraging (a,c,e) and low-density foraging (b,d,f), respectively. The total labelled data set (a,b) as well as single dives (c,d) show diving data (grey), labelled PCE (orange) and predicted PCE (blue and green). Accelerometer traces (e,f) for the individual dives are also shown. The gap in data in (f) is an area where no video was available owing to low ambient light—the data for this part of the dive were removed for the analysis.

Figure 4.

Predicted PCE from two deep learning models (a, CNN; b, V-Net) trained on tri-axial accelerometer data and validated with bird-borne video footage. The predictions are from 15 penguins that were not used in the training of the models. The different coloured points represent individual penguins. Green dashed lines show the linear regression of annotated and predicted PCE per individual dive, while the red solid line shows a 1:1 linear regression.

Figure 5.

Kernel density plots showing the distribution of PCE per dive, as observed from video loggers attached to chinstrap penguins and predicted by two deep learning models (CNN and V-Net). Separate plots are shown for the different dive phases: (a) surface behaviour, (b) descent phase, (c) bottom phase and (d) ascent phase.