Cucumber Seedling Segmentation Network Based on a Multiview Geometric Graph Encoder from 3D Point Clouds

Abstract

Plant phenotyping plays a pivotal role in observing and comprehending the growth and development of plants. In phenotyping, plant organ segmentation based on 3D point clouds has garnered increasing attention in recent years. However, using only the geometric relationship features of Euclidean space still cannot accurately segment and measure plants. To this end, we mine more geometric features and propose a segmentation network based on a multiview geometric graph encoder, called SN-MGGE. First, we construct a point cloud acquisition platform to obtain the cucumber seedling point cloud dataset, and employ CloudCompare software to annotate the point cloud data. The GGE module is then designed to generate the point features, including the geometric relationships and geometric shape structure, via a graph encoder over the Euclidean and hyperbolic spaces. Finally, the semantic segmentation results are obtained via a downsampling operation and multilayer perceptron. Extensive experiments on a cucumber seedling dataset clearly show that our proposed SN-MGGE network outperforms several mainstream segmentation networks (e.g., PointNet++, AGConv, and PointMLP), achieving mIoU and OA values of 94.90% and 97.43%, respectively. On the basis of the segmentation results, 4 phenotypic parameters (i.e., plant height, leaf length, leaf width, and leaf area) are extracted through the K-means clustering method; these parameters are very close to the ground truth, and the R ² values reach 0.98, 0.96, 0.97, and 0.97, respectively. Furthermore, an ablation study and a generalization experiment also show that the SN-MGGE network is robust and extensive.

Fig. 1.

Overview of the proposed method. (A) Top view. (B) Side view. (C) Top view. (D) Side view. (E) Plant height. (F) Calculations of leaf length, width, and leaf area.

Fig. 2.

(A to C) Cucumber planting greenhouse and data collection platform.

Fig. 3.

Sample images of reconstructed cucumber seedling point clouds.

Fig. 4.

Architecture of SN-MGGE for segmentation. Given the input point clouds, the GGE module progressively extracts local geometric features, as well as geometric structure features, across both the Euclidean and hyperbolic spaces. We reduce the number of points progressively and form a hierarchical structure to obtain diverse granularity features by stacking multiple GGE-P modules. The definitive point features are derived through interpolation and repetition, which are then fed into the MLP layer to generate the segmentation results. Note that N is the number of point clouds, K is the number of neighbors of the target point, l denotes the lth layer, λ is a scale factor, and D, d, t, and M denote the dimensions. For a target point x_i, the spatial and feature inputs are Δx_ij = [x_i; x_j − x_i] and Δf_ij = [f_i; f_j − f_i], respectively.

Fig. 5.

Phenotypic parameters, including (A) plant height C_h, (B) leaf length C_l, leaf width C_w, and (C) leaf area C_a, are extracted.

Fig. 6.

The decreasing trend of loss values during the training and validation stages.

Fig. 7.

Semantic segmentation results of the SN-MGGE network, DGCNN, PointNet, and ground truth (left) on 4 different cucumber seedlings (A-D); some plant areas are enlarged to provide more details

Fig. 8.

Individual plant segmentation results of the SN-MGGE network with 3 clustering methods. (A) Ground truth. (B) K-means clustering. (C) MeanShift clustering. (D) Euclidean clustering.

Fig. 9.

Phenotypic parameter comparison of the extracted values and measured values. (A) Leaf length. (B) Leaf width. (C) Plant height. (D) Leaf area.

Fig. 10.

(A to H) Visualization of the plant point clouds with varying removal ratios and numbers of noise.

Fig. 11.

(A) Overall accuracy with different removal ratios. (B) Overall accuracy with different amounts of noise.

Fig. 12.

The qualitative instance segmentation comparison of the 3 plant species: (A) tobacco plant, (B) tomato plant, and (C) sorghum plant. Each segmented plant point cloud generated by SN-MGGE is compared with its corresponding ground truth. The enlarged details are within the wireframe.

Fig. 13.

Qualitative results on ShapeNet Part dataset by visualizing the segmentation results from SN-MGGE across all 16 object categories.