Going deeper in the automated identification of Herbarium specimens

Abstract

Background: Hundreds of herbarium collections have accumulated a valuable heritage and knowledge of plants over several centuries. Recent initiatives started ambitious preservation plans to digitize this information and make it available to botanists and the general public through web portals. However, thousands of sheets are still unidentified at the species level while numerous sheets should be reviewed and updated following more recent taxonomic knowledge. These annotations and revisions require an unrealistic amount of work for botanists to carry out in a reasonable time. Computer vision and machine learning approaches applied to herbarium sheets are promising but are still not well studied compared to automated species identification from leaf scans or pictures of plants in the field.

Results: In this work, we propose to study and evaluate the accuracy with which herbarium images can be potentially exploited for species identification with deep learning technology. In addition, we propose to study if the combination of herbarium sheets with photos of plants in the field is relevant in terms of accuracy, and finally, we explore if herbarium images from one region that has one specific flora can be used to do transfer learning to another region with other species; for example, on a region under-represented in terms of collected data.

Conclusions: This is, to our knowledge, the first study that uses deep learning to analyze a big dataset with thousands of species from herbaria. Results show the potential of Deep Learning on herbarium species identification, particularly by training and testing across different datasets from different herbaria. This could potentially lead to the creation of a semi, or even fully automated system to help taxonomists and experts with their annotation, classification, and revision works.

Keywords: Biodiversity informatics; Computer vision; Deep learning; Herbaria; Plant identification.

Fig. 1

Modified Inception module using PReLU and Batch Normalization

Fig. 2

Ardisia revoluta Kunth herbarium sheet sample taken from Arizona State University Herbarium

Fig. 3

Ten leaf-scan images of different species used in the CRLeaves (CR) dataset: a Acnistus arborescens (L.) Schltdl, b Brunfelsia nitida Benth, c Clusia rosea Jacq, d Dalbergia retusa Hemsl, e Ehretia latifolia Loisel. ex A.DC, f Guazuma ulmifolia Lam, g Malvaviscus arboreus Cav, h Pentas lanceolata (Forssk.) Deflers, i Persea americana Mill, j Piper auritum Kunth

Fig. 4

Ten herbarium sheet images of different species used in the H255 dataset: a Acnistus arborescens (L.) Schltdl, b Brunfelsia nitida Benth, c Clusia rosea Jacq, d Dalbergia retusa Hemsl, e Ehretia latifolia Loisel. ex A.DC, f Guazuma ulmifolia Lam, g Malvaviscus arboreus Cav, h Pentas lanceolata (Forssk.) Deflers, i Persea americana Mill, j Piper auritum Kunth

Fig. 5

Images of different species used in the PlantCLEF (PC) dataset: a Abies alba Mill., b Cirsium oleraceum (L.) Scop., c Datura stramonium L., d Eryngium campestre L., e Gentiana verna L., f Hedera helix L., g Pistacia lentiscus L., h Punica granatum L., i Quercus cerris L., j Scolymus hispanicus L

Fig. 6

Ten herbarium sheet images used in the Herbaria1K (H1K) dataset: a Abies alba Mill, b Cirsium oleraceum (L.) Scop, c Datura stramonium L, d Eryngium campestre L, e Gentiana verna L, f Hedera helix L, g Pistacia lentiscus L, h Punica granatum L, i Quercus cerris L, j Scolymus hispanicus L

Fig. 7

Comparison of losses of R.P C.P C, I.P C.P C and H1K.P C.P C experiments