Exploiting 2D Neural Network Frameworks for 3D Segmentation Through Depth Map Analytics of Harvested Wild Blueberries (Vaccinium angustifolium Ait.)

Abstract

This study introduced a novel approach to 3D image segmentation utilizing a neural network framework applied to 2D depth map imagery, with Z axis values visualized through color gradation. This research involved comprehensive data collection from mechanically harvested wild blueberries to populate 3D and red-green-blue (RGB) images of filled totes through time-of-flight and RGB cameras, respectively. Advanced neural network models from the YOLOv8 and Detectron2 frameworks were assessed for their segmentation capabilities. Notably, the YOLOv8 models, particularly YOLOv8n-seg, demonstrated superior processing efficiency, with an average time of 18.10 ms, significantly faster than the Detectron2 models, which exceeded 57 ms, while maintaining high performance with a mean intersection over union (IoU) of 0.944 and a Matthew's correlation coefficient (MCC) of 0.957. A qualitative comparison of segmentation masks indicated that the YOLO models produced smoother and more accurate object boundaries, whereas Detectron2 showed jagged edges and under-segmentation. Statistical analyses, including ANOVA and Tukey's HSD test (α = 0.05), confirmed the superior segmentation performance of models on depth maps over RGB images (p < 0.001). This study concludes by recommending the YOLOv8n-seg model for real-time 3D segmentation in precision agriculture, providing insights that can enhance volume estimation, yield prediction, and resource management practices.

Keywords: Detectron2; YOLOv8; point clouds; precision agriculture; time of flight.

Figure 1

Example of wild blueberries (Vaccinium angustifolium Ait.) at time of harvest, illustrating the irregular clustering.

Figure 2

Visual demonstration of conversion from point cloud to depth map using the jet colormap as Z axis representation in mm, where the background color of the depth map was set to blue.

Figure 3

Dual camera mount setup for data collection with Basler Blaze-101 (67° by 51° in the X and Y axes, respectively) and Lucid Vision Labs Triton (60° by 46° in the X and Y axes, respectively).

Figure 4

Visualization of segmentation mask correctness of YOLO masks for ToF 3D camera and 2D RGB camera, with true positive as green, true negative as blue, false positive as red, and false negative as orange.

Figure 5

Visualization of segmentation mask correctness of Detectron2 masks for ToF 3D and 2D RGB cameras, with true positive as green, true negative as blue, false positive as red, and false negative as orange.

Figure 6

Sample confusion matrices of Detectron2 R50 with FPN and YOLOv8n-seg on the testing dataset of the depth image dataset.

Figure 7

Sample confusion matrices of Detectron2 R50 with FPN and YOLOv8n-seg on the testing dataset of the RGB image dataset.