Application of a deep learning image classifier for identification of Amazonian fishes

Abstract

Given the sharp increase in agricultural and infrastructure development and the paucity of widespread data available to support conservation management decisions, a more rapid and accurate tool for identifying fish fauna in the world's largest freshwater ecosystem, the Amazon, is needed. Current strategies for identification of freshwater fishes require high levels of training and taxonomic expertise for morphological identification or genetic testing for species recognition at a molecular level. To overcome these challenges, we built an image masking model (U-Net) and a convolutional neural net (CNN) to classify Amazonian fish in photographs. Fish used to generate training data were collected and photographed in tributaries in seasonally flooded forests of the upper Morona River valley in Loreto, Peru in 2018 and 2019. Species identifications in the training images (n = 3068) were verified by expert ichthyologists. These images were supplemented with photographs taken of additional Amazonian fish specimens housed in the ichthyological collection of the Smithsonian's National Museum of Natural History. We generated a CNN model that identified 33 genera of fishes with a mean accuracy of 97.9%. Wider availability of accurate freshwater fish image recognition tools, such as the one described here, will enable fishermen, local communities, and citizen scientists to more effectively participate in collecting and sharing data from their territories to inform policy and management decisions that impact them directly.

Keywords: Neotropical; computer vision; conservation technology; deep machine learning; freshwater fish; species identification.

FIGURE 1

Example of unmasked (left) and masked (right) images of a fish (Bario steindachneri).

FIGURE 2

Confusion matrix visualization of computer vision model validation results. The x‐axis depicts the genus predicted by the model. The y‐axis depicts the actual genus to which the image belongs, organized by taxonomic class, family and genus according to Fricke et al. (2018). Correct identifications are depicted in the left‐to‐right diagonal, with a darker color indicating more correct identifications, and blank yellow squares indicating zeros. Masked image examples on y‐axis are as follows: (a) Bryconops, (b) Tetragonopterus, (c) Astyanax, (d) Moenkhausia, (e) Gymnotus, (f) Ancistrus, (g) Corydoras, and (h) Bujurquina.