Species identification by conservation practitioners using online images: accuracy and agreement between experts

Abstract

Emerging technologies have led to an increase in species observations being recorded via digital images. Such visual records are easily shared, and are often uploaded to online communities when help is required to identify or validate species. Although this is common practice, little is known about the accuracy of species identification from such images. Using online images of newts that are native and non-native to the UK, this study asked holders of great crested newt (Triturus cristatus) licences (issued by UK authorities to permit surveying for this species) to sort these images into groups, and to assign species names to those groups. All of these experts identified the native species, but agreement among these participants was low, with some being cautious in committing to definitive identifications. Individuals' accuracy was also independent of both their experience and self-assessed ability. Furthermore, mean accuracy was not uniform across species (69-96%). These findings demonstrate the difficulty of accurate identification of newts from a single image, and that expert judgements are variable, even within the same knowledgeable community. We suggest that identification decisions should be made on multiple images and verified by more than one expert, which could improve the reliability of species data.

Keywords: Citizen science; Crowdsourced images; Ecological monitoring; Invasive species; Jizz; Newts; Species observations; Validation; Validation..

Figure 1. Individual percentage mean accuracy for correct solution of grouping and naming the four study species, compared with experience (filled circles).

Error bars show ±1 standard error around the mean. Mean accuracy for identification of study species only (open circles) shown for comparison and follows a similar trend.

Figure 2. Average accuracy of participants (grey) in relation to the following factors (white): (A) self-assessed abilities; (B) self-assessed ability in comparison with peers; (C) years of experience in surveying; (D) type of surveying (“Prof”, professional, “ARG”, Amphibian and Reptile Groups, “Other”, affiliations not listed, NARRS, “National Amphibian and Reptile Recording Scheme”, and “None”, no affiliation).

In (D) participants total more than 100% as 12 participants surveyed in more than one capacity. Error bars show ±1 standard error around the mean.

Figure 3. Boxplot showing accuracy rates per species.

Median identification accuracy (horizontal lines in boxes) was highest for images of the target species T. cristatus, then I. alpestris, but lower for the two smaller native newts L. helveticus and L. vulgaris. The range of accurate scores was large in these two smaller newts, especially L. helveticus, and smaller in T. cristatus (with one outlier) and I. alpestris (with four outliers).

Figure 4. Confusion matrix showing consensus between the species name assigned to the downloaded images (columns), and how participants named images (rows).

For each column (n = 340), percentage agreement on species name is shown at the bottom. The greatest consensus is for images of T. cristatus, and there is notable confusion between the two newts of the same genus, L. vulgaris and L. helveticus.

Figure 5. Mean accuracy relating to content and composition of images.

Accuracy was highest in images showing (A) ventral perspective (90%) and (B) whole organism (85%) with no interaction between the two. Mean accuracy was not influenced by the visibility of (C) defined characteristics. Error bars show ±1 standard error around the mean.