The role of individual variability on the predictive performance of machine learning applied to large bio-logging datasets

Abstract

Animal-borne tagging (bio-logging) generates large and complex datasets. In particular, accelerometer tags, which provide information on behaviour and energy expenditure of wild animals, produce high-resolution multi-dimensional data, and can be challenging to analyse. We tested the performance of commonly used artificial intelligence tools on datasets of increasing volume and dimensionality. By collecting bio-logging data across several sampling seasons, datasets are inherently characterized by inter-individual variability. Such information should be considered when predicting behaviour. We integrated both unsupervised and supervised machine learning approaches to predict behaviours in two penguin species. The classified behaviours obtained from the unsupervised approach Expectation Maximisation were used to train the supervised approach Random Forest. We assessed agreement between the approaches, the performance of Random Forest on unknown data and the implications for the calculation of energy expenditure. Consideration of behavioural variability resulted in high agreement (> 80%) in behavioural classifications and minimal differences in energy expenditure estimates. However, some outliers with < 70% of agreement, highlighted how behaviours characterized by signal similarity are confused. We advise the broad bio-logging community, approaching these large datasets, to be cautious when upscaling predictions, as this might lead to less accurate estimates of behaviour and energy expenditure.

Figure 1

Conceptual overview of research questions and approach followed in this study. Both unsupervised and supervised machine learning approaches are applied on large accelerometer datasets to detect and predict behavioural activities. We quantify the behavioural variation included in the training datasets used for the supervised approach and test their predictive performance. We finally validate DEE estimates resulted from the behavioural activity budgets from the two approaches based on energetic validation obtained on Adélie penguin (Pygoscelis adeliae) from Hicks et al..

Figure 2

Example of behavioural classification in Adélie penguins (Pygoscelis adeliae) while diving. (a) diving profiles, (b) Pitch (degrees) indicating body posture while diving, (c) Vectorial Dynamic Body Acceleration (VeDBA, g) indicating overall acceleration while moving underwater. Colours indicate the behavioural states identified while diving.

Figure 3

Example of behavioural classification in Little penguins (Eudyptula minor) while diving. (a) diving profiles, (b) Pitch (degrees) indicating body posture while diving, (c) Vectorial Dynamic Body Acceleration (VeDBA, g) indicating overall acceleration while moving underwater. Colours indicate the behavioural states identified while diving.

Figure 4

Repeatability scores. Values are calculated for pitch (blue) and VeDBA (yellow) on training datasets obtained from Season 1 (squares) and both seasons (triangles) for both species, Adélie penguin (Pygoscelis adeliae) (a) and little penguin (Eudyptula minor) (b).

Figure 5

Agreement between machine learning approaches. Overall agreement between behavioural classification performed by the unsupervised machine learning algorithm Expectation Maximisation (EM) and one returned by the supervised machine learning algorithm Random Forest (RF), ran on data collected during foraging trips of Adélie penguins (Pygoscelis adeliae) (a), and little penguins (Eudyptula minor) (b).

Figure 6

Daily Energy Expenditure estimated using results from two machine learning approaches. Energy Expenditure obtained accounting for the activity budgets predicted by unsupervised machine learning algorithm Expectation Maximisation (EM) and the supervised machine learning algorithm Random Forest (RF) for Adélie penguin (Pygoscelis adeliae). (a) activity budgets used for calculating energy expenditure as in Hicks et al.2020, (b) boxplots of the distributions of the resulting Energy Expenditure, (c) comparison and regression between the Energy Expenditure calculated with the two approaches.