An explainable deep machine vision framework for plant stress phenotyping

Abstract

Current approaches for accurate identification, classification, and quantification of biotic and abiotic stresses in crop research and production are predominantly visual and require specialized training. However, such techniques are hindered by subjectivity resulting from inter- and intrarater cognitive variability. This translates to erroneous decisions and a significant waste of resources. Here, we demonstrate a machine learning framework's ability to identify and classify a diverse set of foliar stresses in soybean [Glycine max (L.) Merr.] with remarkable accuracy. We also present an explanation mechanism, using the top-K high-resolution feature maps that isolate the visual symptoms used to make predictions. This unsupervised identification of visual symptoms provides a quantitative measure of stress severity, allowing for identification (type of foliar stress), classification (low, medium, or high stress), and quantification (stress severity) in a single framework without detailed symptom annotation by experts. We reliably identified and classified several biotic (bacterial and fungal diseases) and abiotic (chemical injury and nutrient deficiency) stresses by learning from over 25,000 images. The learned model is robust to input image perturbations, demonstrating viability for high-throughput deployment. We also noticed that the learned model appears to be agnostic to species, seemingly demonstrating an ability of transfer learning. The availability of an explainable model that can consistently, rapidly, and accurately identify and quantify foliar stresses would have significant implications in scientific research, plant breeding, and crop production. The trained model could be deployed in mobile platforms (e.g., unmanned air vehicles and automated ground scouts) for rapid, large-scale scouting or as a mobile application for real-time detection of stress by farmers and researchers.

Keywords: explainable deep learning; machine learning; plant stress phenotyping; precision agriculture; resolving rater variabilities.

Fig. 1.

Schematic illustration of foliar plant stresses in soybean grouped into two major categories, biotic (bacterial and fungal) and abiotic (nutrient deficiency and chemical injury) stress. The images were used to develop the DCNN for the following eight stresses: bacterial blight (Pseudomonas savastanoi pv. glycinea), bacterial pustule (Xanthomonas axonopodis pv. glycines), sudden death syndrome (SDS, Fusarium virguliforme), Septoria brown spot (Septoria glycines), frogeye leaf spot (Cercospora sojina), IDC, potassium deficiency, and herbicide injury. For each stress, information such as symptom descriptors, areas of appearance, and most commonly mistaken stresses that exhibit similar symptoms are listed. These particular foliar stresses were chosen because of their prevalence and confounding symptoms.

Fig. 2.

Overall schematic of the xPlNet framework: (A) DCNN architecture used. (B) Explanation phase. The concept of isolating the top-K high-resolution feature maps learned by the model based on their localized activation levels was applied to automatically visualize important image features used by the DCNN model.

Fig. 3.

Leaf image examples for each soybean stress correctly identified by the DCNN model. The unsupervised explanation framework is applied to isolate the regions of interest (symptoms) extracted by the DCNN model, which are highly correlated (spatially) with the symptoms marked manually by expert raters.

Fig. 4.

This confusion matrix shows the stress classification results of the DCNN model for eight different stresses and healthy leaves. The overall classification accuracy of the model is 94.13%. The highest confusion among stresses was found among bacterial blight, bacterial pustule, and Septoria brown spot, which can be attributed to the similarities in symptom expression among these stresses.

Fig. 5.

Distributions of spatial correlation between human marked symptoms and machine explanations for four stresses: Septoria brown spot, IDC, Herbicide injury, and SDS. As the distributions are significantly skewed toward high-correlation values, they show the success of the DCNN-based severity estimation framework to correctly identify symptoms for these stresses. Shown are a few examples with different severity classes (machine learning explanations on Left, actual images on Right) using a standard discretized severity scale (0% to 25%: resistant; 25% to 50%: moderately resistant; 50% to 75%: susceptible; and 75% to 100%: highly susceptible).