Multimodal deep learning for cephalometric landmark detection and treatment prediction

Abstract

In orthodontics and maxillofacial surgery, accurate cephalometric analysis and treatment outcome prediction are critical for clinical decision-making. Traditional approaches rely on manual landmark identification, which is time-consuming and subject to inter-observer variability, while existing automated methods typically utilize single imaging modalities with limited accuracy. This paper presents DeepFuse, a novel multi-modal deep learning framework that integrates information from lateral cephalograms, CBCT volumes, and digital dental models to simultaneously perform landmark detection and treatment outcome prediction. The framework employs modality-specific encoders, an attention-guided fusion mechanism, and dual-task decoders to leverage complementary information across imaging techniques. Extensive experiments on three clinical datasets demonstrate that DeepFuse achieves a mean radial error of 1.21 mm for landmark detection, representing a 13% improvement over state-of-the-art methods, with a clinical acceptability rate of 92.4% at the 2 mm threshold. For treatment outcome prediction, the framework attains an overall accuracy of 85.6%, significantly outperforming both conventional prediction models and experienced clinicians. The proposed approach enhances diagnostic precision and treatment planning while providing interpretable visualization of decision factors, demonstrating significant potential for clinical integration in orthodontic and maxillofacial practice.

Keywords: Attention mechanism; Cephalometric analysis; Landmark detection; Multi-modal deep learning; Orthodontics; Treatment outcome prediction.

Fig. 1

Overall architecture of the DeepFuse framework, illustrating modality-specific encoders (left), attention-guided fusion module (center), and dual-task decoders (right). The framework processes multiple input modalities including lateral cephalograms, CBCT volumes, and digital dental models to simultaneously perform landmark detection and treatment outcome prediction.

Fig. 2

Multi-source data preprocessing pipeline illustrating modality-specific preprocessing steps including CLAHE for cephalograms, artifact reduction for CBCT, and mesh simplification for dental models, followed by standardization and registration processes.

Fig. 3

Automated registration flowchart showing landmark-based and intensity-based alignment.

Fig. 4

Architecture of the multi-modal feature extraction and fusion mechanism in DeepFuse. The figure shows modality-specific encoders (left), cross-modal alignment (center), and the attention-guided fusion module (right) with connections to downstream task-specific decoders.

Fig. 5

Comparison of mean detection errors (mm) for critical cephalometric landmarks across different methods. DeepFuse consistently demonstrates lower error rates, with particular improvement for traditionally challenging landmarks (PTM, Go, Or).

Fig. 6

Grad-CAM visualizations for treatment outcome predictions across different case types. Warmer colors indicate regions with greater influence on prediction outcomes.