Image-based dynamic quantification and high-accuracy 3D evaluation of canopy structure of plant populations

Abstract

Background and aims: Global agriculture is facing the challenge of a phenotyping bottleneck due to large-scale screening/breeding experiments with improved breeds. Phenotypic analysis with high-throughput, high-accuracy and low-cost technologies has therefore become urgent. Recent advances in image-based 3D reconstruction offer the opportunity of high-throughput phenotyping. The main aim of this study was to quantify and evaluate the canopy structure of plant populations in two and three dimensions based on the multi-view stereo (MVS) approach, and to monitor plant growth and development from seedling stage to fruiting stage.

Methods: Multi-view images of flat-leaf cucumber, small-leaf pepper and curly-leaf eggplant were obtained by moving a camera around the plant canopy. Three-dimensional point clouds were reconstructed from images based on the MVS approach and were then converted into surfaces with triangular facets. Phenotypic parameters, including leaf length, leaf width, leaf area, plant height and maximum canopy width, were calculated from reconstructed surfaces. Accurate evaluation in 2D and 3D for individual leaves was performed by comparing reconstructed phenotypic parameters with referenced values and by calculating the Hausdorff distance, i.e. the mean distance between two surfaces.

Key results: Our analysis demonstrates that there were good agreements in leaf parameters between referenced and estimated values. A high level of overlap was also found between surfaces of image-based reconstructions and laser scanning. Accuracy of 3D reconstruction of curly-leaf plants was relatively lower than that of flat-leaf plants. Plant height of three plants and maximum canopy width of cucumber and pepper showed an increasing trend during the 70 d after transplanting. Maximum canopy width of eggplants reached its peak at the 40th day after transplanting. The larger leaf phenotypic parameters of cucumber were mostly found at the middle-upper leaf position.

Conclusions: High-accuracy 3D evaluation of reconstruction quality indicated that dynamic capture of the 3D canopy based on the MVS approach can be potentially used in 3D phenotyping for applications in breeding and field management.

Fig. 1.

Workflow of 3D reconstruction and accuracy evaluation for plant structure.

Fig. 2.

Camera positions around the original sparse point clouds of an experimental plant, obtained using VisualSFM. (A) Single-layer photograph of eggplant. (B) Double-layer photograph of cucumber.

Fig. 3.

Sketch map of blade length and leaf width (A), plant height (B) and maximum canopy width (C). (A) The dotted line between 1 and 2 is blade width and that between 3 and 4 is blade length. (B) Plant height is equal to canopy height H₁ minus soil depth H₂. (C) Maximum canopy width is the maximum of canopy widths L₁ and L₂.

Fig. 4.

Point clouds of pepper at the 70th day after transplanting reconstructed from 40 (A), 60 (B) and 80 (C) images.

Fig. 5.

Segmentation of point clouds (A) and 3D surface reconstruction (B) of (from left to right) pepper, eggplant and cucumber at the 40th day after transplanting. Segmentation of point clouds (C) and 3D surface reconstruction (D) of cucumber at the 20th, 40th, 60th and 70th days after transplanting (from left to right). Colours represent different leaf positions and black represents the stem.

Fig. 6.

Visualization of reconstruction accuracy for cucumber (A, B), pepper (C, D) and eggplant (E, F) based on Hausdorff distances at the 40th and 60th days after transplanting. Blue represents high accuracy while red represents relatively low accuracy.

Fig. 7.

Plant height (A) and maximum canopy width (B) of cucumber, pepper and eggplant at the 20th, 40th, 60th and 70th days after transplanting.

Fig. 8.

Leaf area, leaf length and leaf width of cucumber at six growth stages (6-, 8-, 10-, 13-, 15- and 18-leaf stages) of cucumber within 70 d after transplanting. Leaf position was sorted from the first true leaf (position 1) to the top leaf.