Automatic Branch-Leaf Segmentation and Leaf Phenotypic Parameter Estimation of Pear Trees Based on Three-Dimensional Point Clouds

Abstract

The leaf phenotypic traits of plants have a significant impact on the efficiency of canopy photosynthesis. However, traditional methods such as destructive sampling will hinder the continuous monitoring of plant growth, while manual measurements in the field are both time-consuming and laborious. Nondestructive and accurate measurements of leaf phenotypic parameters can be achieved through the use of 3D canopy models and object segmentation techniques. This paper proposed an automatic branch-leaf segmentation pipeline based on lidar point cloud and conducted the automatic measurement of leaf inclination angle, length, width, and area, using pear canopy as an example. Firstly, a three-dimensional model using a lidar point cloud was established using SCENE software. Next, 305 pear tree branches were manually divided into branch points and leaf points, and 45 branch samples were selected as test data. Leaf points were further marked as 572 leaf instances on these test data. The PointNet++ model was used, with 260 point clouds as training input to carry out semantic segmentation of branches and leaves. Using the leaf point clouds in the test dataset as input, a single leaf instance was extracted by means of a mean shift clustering algorithm. Finally, based on the single leaf point cloud, the leaf inclination angle was calculated by plane fitting, while the leaf length, width, and area were calculated by midrib fitting and triangulation. The semantic segmentation model was tested on 45 branches, with a mean Precision_sem, mean Recall_sem, mean F1-score, and mean Intersection over Union (IoU) of branches and leaves of 0.93, 0.94, 0.93, and 0.88, respectively. For single leaf extraction, the Precision_ins, Recall_ins, and mean coverage (mCoV) were 0.89, 0.92, and 0.87, respectively. Using the proposed method, the estimated leaf inclination, length, width, and area of pear leaves showed a high correlation with manual measurements, with correlation coefficients of 0.94 (root mean squared error: 4.44°), 0.94 (root mean squared error: 0.43 cm), 0.91 (root mean squared error: 0.39 cm), and 0.93 (root mean squared error: 5.21 cm²), respectively. These results demonstrate that the method can automatically and accurately measure the phenotypic parameters of pear leaves. This has great significance for monitoring pear tree growth, simulating canopy photosynthesis, and optimizing orchard management.

Keywords: leaf phenotype; pear canopy; point cloud segmentation.

Figure 1

Workflow of this study: (a) Data acquisition; (b) Data pre-processing and branch level dataset construction; (c) Branch–leaf segmentation with PointNet++ segmentation model; (d) Single leaf segmentation with Mean Shift Clustering Model; (e) Leaf phenotypic trait extraction.

Figure 2

Structure of PointNet++ segmentation network. N represents the number of points, K represents the number of groups, d represents the coordinate dimension, and C represents the feature dimension.

Figure 3

Schematic diagram of phenotypic parameters estimation based on single leaf point cloud. (a) Original leaf point cloud; (b) Leaf point cloud after smoothing using MLS; (c) Leaf point cloud plane fitting; (d) Leaf inclination angle estimation; (e) Leaf length and width estimation; (f) Leaf area estimation. S_leaf in (c,d) is the leaf point cloud fitting plane.

Figure 4

Schematic diagram of estimating leaf length and width based on midrib fitting. (a) The leaf base point P and the leaf tip point Q; (b) Approximate point cloud (purple points) of midrib; (c) The approximate midrib point cloud is projected onto the plane S_vein; (d) Midrib fitting point cloud (red points) after projection; (e) The starting point M and the ending point L when estimating the leaf width; (f) Approximate point cloud (dark purple points) of estimating leaf width; (g) The approximate point cloud of estimating leaf width is projected onto the plane S; (h) Fitting point cloud for estimating leaf length (red points) and leaf width (blue points). S_vein in (c) is the projection plane of midrib point cloud, and S in (g) is the widest cross section of the leaf.

Figure 5

Visualization of branch–leaf semantic segmentation of branches with different attributes in the test dataset using PointNet++. In each subgraph, the left side shows the manual labeling, the middle shows the model prediction (branch and leaf points are in blue and red, respectively), and the right side shows the difference between them (same and different points of classification are in black and green, respectively).

Figure 6

Distribution of the branch–leaf segmentation evaluation metrics with different branch lengths and numbers of leaves. Each subfigure shows mean Precision_sem, mean Recall_sem, mean F1-score, and mean IoU of each sample with different branch length (subfigures (a–d)) and leaf number (subfigures (e–h)), respectively.

Figure 7

Examples of single leaf segmentation with different radius using mean shift clustering algorithms.

Figure 8

Visualization of single leaf segmentation of branches with different attributes using mean shift clustering algorithm (radius: 45 mm). In each subgraph, the left and right sides are the result of manual and automatic segmentation, respectively. Different leaves are represented by different colors.

Figure 9

Distribution of the single leaf segmentation evaluation metrics with branch length and number of leaves. Each subfigure shows Precision_ins, Recall_ins, and mean mCov of each sample with different branch length (subfigures (a–c)) and leaf number (subfigures (d–f)), respectively.

Figure 10

Comparison of phenotypic parameters estimated by the proposed method with the measured values: (a) leaf inclination angle; (b) leaf length; (c) leaf width; and (d) leaf area.