Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Aug 18;22(16):6210.
doi: 10.3390/s22166210.

An Efficient Ensemble Deep Learning Approach for Semantic Point Cloud Segmentation Based on 3D Geometric Features and Range Images

Muhammed Enes Atik  1 , Zaide Duran  1
Affiliations

An Efficient Ensemble Deep Learning Approach for Semantic Point Cloud Segmentation Based on 3D Geometric Features and Range Images

Muhammed Enes Atik et al. Sensors (Basel). .

Abstract

Mobile light detection and ranging (LiDAR) sensor point clouds are used in many fields such as road network management, architecture and urban planning, and 3D High Definition (HD) city maps for autonomous vehicles. Semantic segmentation of mobile point clouds is critical for these tasks. In this study, we present a robust and effective deep learning-based point cloud semantic segmentation method. Semantic segmentation is applied to range images produced from point cloud with spherical projection. Irregular 3D mobile point clouds are transformed into regular form by projecting the clouds onto the plane to generate 2D representation of the point cloud. This representation is fed to the proposed network that produces semantic segmentation. The local geometric feature vector is calculated for each point. Optimum parameter experiments were also performed to obtain the best results for semantic segmentation. The proposed technique, called SegUNet3D, is an ensemble approach based on the combination of U-Net and SegNet algorithms. SegUNet3D algorithm has been compared with five different segmentation algorithms on two challenging datasets. SemanticPOSS dataset includes the urban area, whereas RELLIS-3D includes the off-road environment. As a result of the study, it was demonstrated that the proposed approach is superior to other methods in terms of mean Intersection over Union (mIoU) in both datasets. The proposed method was able to improve the mIoU metric by up to 15.9% in the SemanticPOSS dataset and up to 5.4% in the RELLIS-3D dataset.

Keywords: autonomous driving; deep learning; light detection and ranging (LiDAR); point cloud; semantic segmentation.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Workflow of the study.
Figure 2
Figure 2
An illustration of the point cloud segment transformed to range image. Red point is center and gray points are neighbor points.
Figure 3
Figure 3
The captured point cloud data is projected to the 2D plane due to LiDAR parameters. Objects close to the sensor are denser, and the density decreases as you move away from the sensor. Some projected objects are marked with red and yellow rectangles.
Figure 4
Figure 4
Addition of weights of two streams.
Figure 5
Figure 5
An illustration of a SegUnet3D architecture. The 64 × 1024 image is in two streams, downsampling in the encoder and then upsampling in the decoder. Thus, the input and output size will be the same. The specified numbers represent the width of the image in that layer.
Figure 6
Figure 6
Qualitative results of the methods for SemanticPOSS. (a) Ground Truth; (b) SegUNet3D; (c) SegNet; (d) U-Net; (e) SqueezeSegV2; (f) PointSeg; (g) SalsaNext.
Figure 7
Figure 7
Semantic segmentation results of the SemanticPOSS dataset are presented as point clouds. (a) Ground Truth; (b) SegUNet3D; (c) SegNet; (d) U-Net; (e) SqueezeSegV2; (f) PointSeg; (g) SalsaNext.
Figure 8
Figure 8
Qualitative results of the methods for RELLLIS-3D. (a) Ground Truth; (b) SegUNet3D; (c) SegNet; (d) U-Net; (e) SqueezeSegV2; (f) PointSeg; (g) SalsaNext.
Figure 9
Figure 9
Semantic segmentation results of the RELLIS-3D dataset are presented as point clouds. (a) Ground Truth; (b) SegUNet3D; (c) SegNet; (d) U-Net; (e) SqueezeSegV2; (f) PointSeg; (g) SalsaNext.
See this image and copyright information in PMC

References

    1. Wu B., Zhou X., Zhao S., Yue X., Keutzer K. SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud; Proceedings of the 2019 International Conference on Robotics and Automation (ICRA); Montreal, QC, Canada. 20–24 May 2019; - DOI
    1. Biasutti P., Lepetit V., Aujol J.F., Bredif M., Bugeau A. LU-net: An efficient network for 3D LiDAR point cloud semantic segmentation based on end-to-end-learned 3D features and U-net; Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops; Seoul, Korea. 27–28 October 2019; - DOI
    1. Li S., Liu Y., Gall J. Rethinking 3-D LiDAR Point Cloud Segmentation. IEEE Trans. Neural Netw. Learn. Syst. 2021 doi: 10.1109/TNNLS.2021.3132836. - DOI - PubMed
    1. Li Y., Ma L., Zhong Z., Liu F., Chapman M.A., Cao D., Li J. Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review. IEEE Trans. Neural Netw. Learn. Syst. 2021;32:3412–3432. doi: 10.1109/TNNLS.2020.3015992. - DOI - PubMed
    1. Nagy B., Benedek C. 3D CNN-based semantic labeling approach for mobile laser scanning data. IEEE Sens. J. 2019;19:10034–10045. doi: 10.1109/JSEN.2019.2927269. - DOI