Towards the fully automated monitoring of ecological communities

Abstract

High-resolution monitoring is fundamental to understand ecosystems dynamics in an era of global change and biodiversity declines. While real-time and automated monitoring of abiotic components has been possible for some time, monitoring biotic components-for example, individual behaviours and traits, and species abundance and distribution-is far more challenging. Recent technological advancements offer potential solutions to achieve this through: (i) increasingly affordable high-throughput recording hardware, which can collect rich multidimensional data, and (ii) increasingly accessible artificial intelligence approaches, which can extract ecological knowledge from large datasets. However, automating the monitoring of facets of ecological communities via such technologies has primarily been achieved at low spatiotemporal resolutions within limited steps of the monitoring workflow. Here, we review existing technologies for data recording and processing that enable automated monitoring of ecological communities. We then present novel frameworks that combine such technologies, forming fully automated pipelines to detect, track, classify and count multiple species, and record behavioural and morphological traits, at resolutions which have previously been impossible to achieve. Based on these rapidly developing technologies, we illustrate a solution to one of the greatest challenges in ecology: the ability to rapidly generate high-resolution, multidimensional and standardised data across complex ecologies.

Keywords: community ecology; computer vision; deep learning; high-resolution monitoring; remote sensing.

FIGURE 1

The automation workflow for monitoring populations and communities. From data collection to the extraction of ecological knowledge, a synthesis of the technologies that can automate the acquisition of information regarding individual traits and species abundances, distributions and interactions, which are key metrics for the monitoring of ecological communities.

FIGURE 2

A diversity of automatic recorders to monitor ecological communities non‐invasively and remotely. (1) Vocalising birds being monitored by microphones deployed on trees. (2) Stridulating and drumming fishes being recorded by hydrophones attached to moorings. (3) Walking elephants producing ground vibrations perceived by geophones. (4) Fish shoal being detected by a sonar. (5) Oceanic glider navigating an Environmental Sampling Processor (ESP) to sample eDNA. (6) Bear being detected by camera traps fixed on trees. (7) Hyperspectral camera mounted on a drone and monitoring tree composition in a forest. (8) LiDAR sensor mounted on an unmanned aerial vehicle monitoring multiple forest canopies. (9) Imaging flow cytometer attached to a mooring and recording planktonic communities. (10) Racoons being detected by thermal and IR cameras at night. (11) Stationary radar and a satellite radar, respectively, monitoring bird and large mammal populations. Recorder's ability to detect the presence of living organisms, count their numbers, classify them at the species level and measure their traits (e.g. behavioural, functional and morphological traits) is evaluated from 1 to 3 levels as follows: 1 bar corresponds to ‘in corner‐case situations only’, 2 bars corresponds to ‘in specific conditions and on specific organisms (for detecting, counting and classifying) or for a limited number of features (for measuring)’, and 3 bars corresponds to ‘in most cases and for most organisms (for detecting, counting and classifying) and for several features (for measuring)’.

FIGURE 3

Deep neural networks and their application in monitoring ecological communities. (a) Schematic representation of a convolutional neural network (CNN) architecture and its application to classify multiple species based on sound or image data. (b) Typical example of CNN output when used to count the number of organisms present in an image such as in (Lu et al., 2019). (c) Typical example of CNN output when used to monitor plant status such as in (Mohanty et al., 2016). (d) and (e) represent the output from other types of deep neural networks (i.e. non‐CNN) used to measure organism morphometrical traits such as in (Jung, 2021) and estimate animal pose such as in (Lauer et al., ; Mathis et al., ; Nath et al., 2019) respectively. Photo credits: Marc Besson.

FIGURE 4

Overview of a fully automated workflow towards the monitoring of multidimensional data from multispecies protist communities in experimental systems. (a) Robotic gantry navigating a microscope and camera over experimental microcosms. (b) Examples of other microcosm landscapes that can be used within this workflow. (c) Video analysis workflow, from raw frames to measurement and classification of moving objects using the CNN classifier. Red bounding boxes indicate the detected individuals and coloured overlay indicate different species. (d–g) Length and width, velocity, classification and trajectory measurements, respectively, obtained by this automated workflow for a single moving object (i.e. protist organism) over the duration of the video. Photo credits: Marc Besson.

FIGURE 5

Insights from real‐time, fully automated in situ monitoring of plants and pollinator interactions. (a) The automated pollinator monitoring system records a green roof comprising Sedum flowers. (b) Continuous surveillance allows the annual phenology of different pollinator groups to be quantified at fine temporal resolutions (blue = honeybees; dark purple = bumblebees; light orange = hoverflies; abundance = number of individual tracks). (c) Diurnal phenology can also be compared across groups, showing a relative preference of hoverflies for mornings and honeybees for evenings. (d) Image from day 234 of 2020, a day of high pollinator activity. (e) Activity of different insect groups on day 234 can be mapped to inflorescences in (d) to quantify plant–pollinator interactions. (f) Real‐time monitoring even allows exploration of pollinator‐pollinator interactions; the activity (total detections) of honeybees, bumblebees and hoverflies is shown for 10‐min intervals during day 234, where bumblebees are only active during a remarkably short period of the day.

FIGURE 6

Futurist examples of fully automated wildlife monitoring programs. (a) Autonomous and wireless underwater vehicle equipped with multiple high‐resolution cameras and hydrophone array, together monitoring multidimensional data about coral reef communities such as habitat complexity, coral species distribution and fish functional diversity. (b) Autonomous and self‐charging drones equipped with LiDAR and hyperspectral cameras for the monitoring of plant and tree flowering phenology.