Multi-feature machine learning model for automatic segmentation of green fractional vegetation cover for high-throughput field phenotyping

Abstract

Background: Accurately segmenting vegetation from the background within digital images is both a fundamental and a challenging task in phenotyping. The performance of traditional methods is satisfactory in homogeneous environments, however, performance decreases when applied to images acquired in dynamic field environments.

Results: In this paper, a multi-feature learning method is proposed to quantify vegetation growth in outdoor field conditions. The introduced technique is compared with the state-of the-art and other learning methods on digital images. All methods are compared and evaluated with different environmental conditions and the following criteria: (1) comparison with ground-truth images, (2) variation along a day with changes in ambient illumination, (3) comparison with manual measurements and (4) an estimation of performance along the full life cycle of a wheat canopy.

Conclusion: The method described is capable of coping with the environmental challenges faced in field conditions, with high levels of adaptiveness and without the need for adjusting a threshold for each digital image. The proposed method is also an ideal candidate to process a time series of phenotypic information throughout the crop growth acquired in the field. Moreover, the introduced method has an advantage that it is not limited to growth measurements only but can be applied on other applications such as identifying weeds, diseases, stress, etc.

Keywords: Field Scanalyzer; Field phenotyping; Fractional cover; Learning-based segmentation; RGB images.

Fig. 1

Schematic representation of the method

Fig. 2

Field Scanalyzer. (Left) The Field Scanalyzer phenotyping platform at Rothamsted Research showing (Right) the cameras within the camera bay directed down, perpendicular to the ground

Fig. 3

Training patches. Examples images from foreground (FG) and background (BG) training patches in various illumination conditions

Fig. 4

Examples of test images and their corresponding ground truths. The test images randomly selected from the image dataset in different illumination conditions. a original image, b reference image segmented manually, c binary image of the reference image

Fig. 5

Multi-feature versus single feature. Comparison of segmented images of supervised learning model with single colour space versus multiple colour spaces

Fig. 6

Comparison of the mean accuracy rate ($Q_{\mathit{seg}}$, $S_r$, and $E_s$). Comparison of different approaches by segmentation quality for ExG, ExGR, CIVE, ACE, K-means, and the proposed method, MFL. The bar indicates the standard deviations

Fig. 7

An example of vegetation segmentation by ExG, ExGR, CIVE, ACE, K-means, and our method

Fig. 8

Digital images of a single section of a wheat plot (Triticum aesvtivum L. cv. Soissons) and the vegetation extracted using various image segmentation methods. Images were captured 165 DAS at a 10:16 AM, b 12:44 PM, c 3:36 PM, d 5:03 PM

Fig. 9

Comparison of manual canopy cover estimates of 54 wheat plots determined using leaf area index (LAI) with the automatic methods, ExG, ExGR, CIVE, ACE, K-means, and MFL

Fig. 10

Canopy cover estimates. Comparison of the segmentation results for canopy cover at 33 random time points of the UK 2015–2016 growing season. a Crusoe, b Gatsby, c Widgeon

Fig. 11

Segmentation results of six methods. The columns from the first to sixth demonstrate the segmentation results by ExG, ExGR, CIVE, K-means, ACE, and the proposed MFL method respectively. a Crusoe 213 DAS, b Gatsby 262 DAS, c Widgeon 230 DAS