Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Feb:11596:115962B.
doi: 10.1117/12.2582205. Epub 2021 Feb 15.

FlyBy CNN: A 3D surface segmentation framework

Louis Boumbolo  1 , Maxime Dumont  2 , Serge Brosset  2 , Jonas Bianchi  2 , Antonio Ruellas  2 , Marcela Gurgel  2 , Camila Massaro  2 , Aron Aliaga Del Castillo  2 , Marcos Ioshida  2 , Marilia Yatabe  2 , E Benavides  2 , Hector Rios  2 , Fabiana Soki  2 , Gisele Neiva  2 , Beatriz Paniagua  3 , Lucia Cevidanes  2 , Martin Styner  1 , Juan C Prieto  1
Affiliations

FlyBy CNN: A 3D surface segmentation framework

Louis Boumbolo et al. Proc SPIE Int Soc Opt Eng. 2021 Feb.

Abstract

In this paper, we present FlyBy CNN, a novel deep learning based approach for 3D shape segmentation. FlyByCNN consists of sampling the surface of the 3D object from different view points and extracting surface features such as the normal vectors. The generated 2D images are then analyzed via 2D convolutional neural networks such as RUNETs. We test our framework in a dental application for segmentation of intra-oral surfaces. The RUNET is trained for the segmentation task using image pairs of surface features and image labels as ground truth. The resulting labels from each segmented image are put back into the surface thanks to our sampling approach that generates 1-1 correspondence of image pixels and triangles in the surface model. The segmentation task achieved an accuracy of 0.9.

Keywords: intra oral surface; mesh; segmentation; shape analysis.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Octree rendering. A line segment is intersected with the octree to quickly locate the corresponding triangles of the 3D surface.
Figure 2.
Figure 2.
a) Icosahedron subdivision and the tangent plane. b) Mesh features. c) Labeled regions.
Figure 3.
Figure 3.
a) Feature image generation via octrees and surface sampling. b) Image segmentation via RUNET. c) The resulting labels from the segmentation are put back in the 3D surface by using the intersections found during the sampling.
Figure 4.
Figure 4.
Post processing of the mesh to isolate dental crowns. The sampling via FlyBy approach may not sample all triangles of the mesh. a) Majority voting output for each individual snapshot. b) Removal of unlabeled triangles. c) Region growing to label each tooth with a unique label. d) Erosion of the boundary between teeth and gum.
Figure 5.
Figure 5.
Prediction on 10 evaluation scans for the gum, teeth, and boundary between gum and teeth. The predicted boundary is used to split each tooth and produce a single labeled output for each crown.
Figure 6.
Figure 6.
Resulting labeled IOS from our test set. Top row, lower arches. Bottom row, upper arches.
See this image and copyright information in PMC

References

    1. Berrendero S, Salido MP, Valverde A, Ferreiroa A, and Pradíes G. Influence of conventional and digital intraoral impressions on the fit of cad/cam-fabricated all-ceramic crowns. Clinical oral investigations, 20(9):2403–2410, 2016. - PubMed
    1. Li Zhongyi and Wang Hao. Interactive tooth separation from dental model using segmentation field. PloS one, 11(8):e0161159, 2016. - PMC - PubMed
    1. Ronneberger Olaf, Fischer Philipp, and Brox Thomas. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
    1. He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
    1. Aubry Mathieu, Schlickewei Ulrich, and Cremers Daniel. The wave kernel signature: A quantum mechanical approach to shape analysis. In 2011 IEEE international conference on computer vision workshops (ICCV workshops), pages 1626–1633. IEEE, 2011.