Deep ensemble learning-driven fully automated multi-structure segmentation for precision craniomaxillofacial surgery

Abstract

Objectives: Accurate segmentation of craniomaxillofacial (CMF) structures and individual teeth is essential for advancing computer-assisted CMF surgery. This study developed CMF-ELSeg, a novel fully automatic multi-structure segmentation model based on deep ensemble learning.

Methods: A total of 143 CMF computed tomography (CT) scans were retrospectively collected and manually annotated by experts for model training and validation. Three 3D U-Net-based deep learning models (V-Net, nnU-Net, and 3D UX-Net) were benchmarked. CMF-ELSeg employed a coarse-to-fine cascaded architecture and an ensemble approach to integrate the strengths of these models. Segmentation performance was evaluated using Dice score and Intersection over Union (IoU) by comparing model predictions to ground truth annotations. Clinical feasibility was assessed through qualitative and quantitative analyses.

Results: In coarse segmentation of the upper skull, mandible, cervical vertebra, and pharyngeal cavity, 3D UX-Net and nnU-Net achieved Dice scores above 0.96 and IoU above 0.93. For fine segmentation and classification of individual teeth, the cascaded 3D UX-Net performed best. CMF-ELSeg improved Dice scores by 3%-5% over individual models for facial soft tissue, upper skull, mandible, cervical vertebra, and pharyngeal cavity segmentation, and maintained high accuracy Dice > 0.94 for most teeth. Clinical evaluation confirmed that CMF-ELSeg performed reliably in patients with skeletal malocclusion, fractures, and fibrous dysplasia.

Conclusion: CMF-ELSeg provides high-precision segmentation of CMF structures and teeth by leveraging multiple models, serving as a practical tool for clinical applications and enhancing patient-specific treatment planning in CMF surgery.

Keywords: computed tomography; craniomaxillofacial surgery; deep learning; segmentation; virtual surgical planning.

FIGURE 1

Overview of the study design.

FIGURE 2

Architecture of our proposed deep learning model.

FIGURE 3

Quantitative analysis results for segmentation performance. (A) Dice scores for segmentation performance of CMF structures using V-Net, nnU-Net, and 3D UX-Net. (B) Dice scores for segmentation performance of individual teeth using cascaded segmentation networks based on V-Net, nnU-Net, and 3D UX-Net.

FIGURE 4

Segmentation performance of CMF-ELSeg. (A) Quantitative analysis results for segmentation performance of CMF structures and individual teeth. (B) Comparison of segmentation performance between CMF-ELSeg and the baseline models. *P < 0.05.

FIGURE 5

Segmentation and 3D reconstruction results of CMF structures and individual teeth using CMF-ELSeg. (A,B) Segmentation results illustrated for two representative cases. (C,D) 3D reconstruction results illustrated for two representative cases. Case 1: a skeletal class III malocclusion patient with orthodontic brackets. Case 2: a patient who has undergone orthognathic surgery.

FIGURE 6

Segmentation results of individual teeth using CMF-ELSeg and individual cascaded segmentation network. Case 1: a skeletal class III malocclusion patient with orthodontic brackets. Case 2: a patient who has undergone orthognathic surgery.

FIGURE 7

3D reconstruction results and surface deviations of individual teeth using CMF-ELSeg and individual cascaded segmentation network. Case 1: a skeletal class III malocclusion patient with orthodontic brackets. Case 2: a patient who has undergone orthognathic surgery.

FIGURE 8

Clinical feasibility evaluation of CMF-ELSeg. (A) An example of the segmentation and reconstruction results using CMF-ELSeg for patients with skeletal malocclusion. (B) The qualitative analysis results of CMF-ELSeg in Cohort 2. (C) Quantitative analysis results of CMF-ELSeg in Cohort 2. (D) The composition of patients in Cohort 3. (E) The qualitative analysis results of CMF-ELSeg in Cohort 3. (F) Segmentation and reconstruction cases of CMF-ELSeg in Cohort 3.