Hyperspectral Technique Combined With Deep Learning Algorithm for Prediction of Phenotyping Traits in Lettuce

Abstract

The currently available methods for evaluating most biochemical traits of plant phenotyping are destructive and have extremely low throughput. However, hyperspectral techniques can non-destructively obtain the spectral reflectance characteristics of plants, which can provide abundant biophysical and biochemical information. Therefore, plant spectra combined with machine learning algorithms can be used to predict plant phenotyping traits. However, the raw spectral reflectance characteristics contain noise and redundant information, thus can easily affect the robustness of the models developed via multivariate analysis methods. In this study, two end-to-end deep learning models were developed based on 2D convolutional neural networks (2DCNN) and fully connected neural networks (FCNN; Deep2D and DeepFC, respectively) to rapidly and non-destructively predict the phenotyping traits of lettuces from spectral reflectance. Three linear and two nonlinear multivariate analysis methods were used to develop models to weigh the performance of the deep learning models. The models based on multivariate analysis methods require a series of manual feature extractions, such as pretreatment and wavelength selection, while the proposed models can automatically extract the features in relation to phenotyping traits. A visible near-infrared hyperspectral camera was used to image lettuce plants growing in the field, and the spectra extracted from the images were used to train the network. The proposed models achieved good performance with a determination coefficient of prediction ( $R_p^2$ ) of 0.9030 and 0.8490 using Deep2D for soluble solids content and DeepFC for pH, respectively. The performance of the deep learning models was compared with five multivariate analysis method. The quantitative analysis showed that the deep learning models had higher $R_p^2$ than all the multivariate analysis methods, indicating better performance. Also, wavelength selection and different pretreatment methods had different effects on different multivariate analysis methods, and the selection of appropriate multivariate analysis methods and pretreatment methods increased more time and computational cost. Unlike multivariate analysis methods, the proposed deep learning models did not require any pretreatment or dimensionality reduction and thus are more suitable for application in high-throughput plant phenotyping platforms. These results indicate that the deep learning models can better predict phenotyping traits of plants using spectral reflectance.

Keywords: SSC; deep learning; hyperspectral imaging; lettuce; pH; plant phenotyping.

Figure 1

Picture of plants and LQ-FieldPheno in the field.

Figure 2

The process of plant spectra extraction. (A) Calibrated hyperspectral images, (B) binary plant mask, (C) masked RGB plant, (D) 256 bands corresponding to the ROI, and (E) extracted plant mean reflectance spectrum.

Figure 3

Architecture of the naïve version Inception module (A) and Deep2D (B).

Figure 4

FCNN schematic chart (A) and architecture of the DeepFC (B).

Figure 5

Mean spectral reflectance of plants. (A) Raw spectra and spectra pretreated by MWS (B), FDR (C), and SDR (D).

Figure 6

Mean spectral reflectance of plants of different SSC (A) and pH (B).

Figure 7

Prediction of SSC based on Deep2D (A) and DeepFC (B), and pH based on Deep2D (C) and DeepFC (D).

Figure 8

Prediction of SSC based on LWR and SDR (A), and pH based on LWR and MWS (B).