A Swin Transformer-based model for mosquito species identification

Abstract

Mosquito transmit numbers of parasites and pathogens resulting in fatal diseases. Species identification is a prerequisite for effective mosquito control. Existing morphological and molecular classification methods have evitable disadvantages. Here we introduced Deep learning techniques for mosquito species identification. A balanced, high-definition mosquito dataset with 9900 original images covering 17 species was constructed. After three rounds of screening and adjustment-testing (first round among 3 convolutional neural networks and 3 Transformer models, second round among 3 Swin Transformer variants, and third round between 2 images sizes), we proposed the first Swin Transformer-based mosquito species identification model (Swin MSI) with 99.04% accuracy and 99.16% F1-score. By visualizing the identification process, the morphological keys used in Swin MSI were similar but not the same as those used by humans. Swin MSI realized 100% subspecies-level identification in Culex pipiens Complex and 96.26% accuracy for novel species categorization. It presents a promising approach for mosquito identification and mosquito borne diseases control.

Figure 1

The Framework of Swin MSI. (a)The basic architecture for mosquito features extraction and identification. Attention visualization generated by filters at each layer are shown. (b) Details for Swin Transformer block. (c) For mosquito within our dataset 17 species, output is the top 5 confidence species. (d) For mosquito beyond 17 species (defined as novel species), whether the output is a species or a genus is decided after comparing with confidence threshold.

Figure 2

Taxonomic status and index of mosquito species included in this study Both male and female mosquitoes were photographed from different angles such as dorsal, left side, right side, ventral side, etc. Except for 5 species, each mosquito includes 300 images of both sexes, and the resolution of mosquito photos were 4464 × 2976. Cx. pipiens quinquefasciatus, Cx. pipiens pallens, and Cx. pipiens molestus (subspecies level, in dark gray background) were 3 subspecies in Cx. pipiens Complex (species level).

Figure 3

Flowchart of testing deep learning model for intelligent identification of mosquito species.

Figure 4

Comparison of mosquito recognition effects of computer vision network models and variants. (a) Comparison of mosquito identification accuracy between 3 CNNs and 3 Transformer; (b) The best effect CNN (YOLOv5) training set loss curve(blue), validation set loss curve(green) and validation set accuracy curve(orange); (c) The best effect Transformer (Swin Transformer) training set loss curve, validation set loss curve and validation set accuracy curve. (d) Swin-MSI-T test result confusion matrix; (e) Swin-MSI -B test result confusion matrix; (f) Swin-MSI -L test result confusion matrix. Confusion matrix of mosquito labels in which odd numbers represent females and even numbers represent males. The small squares in the confusion matrix represent the recognition readiness rate, from red to green, the recognition readiness rate is getting higher and higher An. sinensis: 1, 2; Cx. pipiens quinquefasciatus: 3, 4; Cx. pipiens pallens: 5, 6; Cx. pipiens molestus: 7,8 Cx. modestus: 9,10; Ae. albopictus: 11, 12 Ae. aegypti: 13, 14; Cx. pallidothorax: 15, 16 Ae. galloisi: 17,18 Ae. vexans: 19, 20; Ae. koreicus: 21, 22 Armigeres subalbatus: 23, 24; Coquillettidia ochracea: 25, 26 Cx. gelidus: 27, 28 Cx. triraeniorhynchus: 29, 30 Mansonia uniformis: 31, 32 An. vagus: 33, 34 Ae. elsaie: 35,36 Toxorhynchites splendens: 37, 38.

Figure 5

Attention visualization of representative mosquitoes of the genera Ae., Cx., An., Armigeres, Coquillettidia and Mansonia. This is a visualization for identifying the regions in the image that can explain the classification progress. Images of Toxorhynchites contain only males, with obvious differences in morphological characteristics, are not shown.