Automatic identification of clinically important Aspergillus species by artificial intelligence-based image recognition: proof-of-concept study

Abstract

While morphological examination is the most widely used for Aspergillus identification in clinical laboratories, PCR-sequencing and MALDI-TOF MS are emerging technologies in more financially-competent laboratories. However, mycological expertise, molecular biologists and/or expensive equipment are needed for these. Recently, artificial intelligence (AI), especially image recognition, is being increasingly employed in medicine for fast and automated disease diagnosis. We explored the potential utility of AI in identifying Aspergillus species. In this proof-of-concept study, using 2813, 2814 and 1240 images from four clinically important Aspergillus species for training, validation and testing, respectively; the performances and accuracies of automatic Aspergillus identification using colonial images by three different convolutional neural networks were evaluated. Results demonstrated that ResNet-18 outperformed Inception-v3 and DenseNet-121 and is the best algorithm of choice because it made the fewest misidentifications (n = 8) and possessed the highest testing accuracy (99.35%). Images showing more unique morphological features were more accurately identified. AI-based image recognition using colonial images is a promising technology for Aspergillus identification. Given its short turn-around-time, minimal demand of expertise, low reagent/equipment costs and user-friendliness, it has the potential to serve as a routine laboratory diagnostic tool after the database is further expanded.

Keywords: Aspergillus; artificial intelligence; automation; identification; image recognition; machine learning.

Figure 1.

Workflow of image classification using convolutional neural networks (CNNs) in this study. (a) General flow of classification using CNN and associated mathematical functions and optimizer. (b) Simplified illustration of fungal identification using CNN. (c) Demonstration of convolution: a matrix operation. (d) Demonstration of max-pooling operation.

Figure 2.

Examples of image data for the four pathogenic Aspergillus species included in the training database. (a) A. flavus NRRL 1957^T. (b) A. fumigatus NRRL 163^T. (c) A. niger NRRL 326^T. (d) A. terreus NRRL 255^T.

Figure 3.

Network training, validation and testing by DenseNet-201. (a) Change in function losses and (b) accuracies throughout the training and validation processes. (c) Heatmap showing the final testing results. t-Distributed Stochastic Neighbour Embedding (t-SNE) visualization of Aspergillus images (d) before and (e) after machine learning through DenseNet-201.

Figure 4.

Network training, validation and testing by Inception-v3. (a) Change in function losses and (b) accuracies throughout the training and validation processes. (c) Heatmap showing the final testing results. t-Distributed Stochastic Neighbour Embedding (t-SNE) visualization of Aspergillus images (d) before and (e) after machine learning through Inception-v3.

Figure 5.

Network training, validation and testing by ResNet-18. (a) Change in function losses and (b) accuracies throughout the training and validation processes. (c) Heatmap showing the final testing results. t-Distributed Stochastic Neighbour Embedding (t-SNE) visualization of Aspergillus images (d) before and (e) after machine learning through ResNet-18.

Figure 6.

Images always misidentified by the three trained networks during testing. Image identifiers (a) 1,628 (A. fumigatus NRRL 163^T day 7) and (b) 1,629 (A. fumigatus NRRL 163^T day 7).