Prediction of midpalatal suture maturation stage based on transfer learning and enhanced vision transformer

Abstract

Background: Maxillary expansion is an important treatment method for maxillary transverse hypoplasia. Different methods of maxillary expansion should be carried out depending on the midpalatal suture maturation levels, and the diagnosis was validated by palatal plane cone beam computed tomography (CBCT) images by orthodontists, while such a method suffered from low efficiency and strong subjectivity. This study develops and evaluates an enhanced vision transformer (ViT) to automatically classify CBCT images of midpalatal sutures with different maturation stages.

Methods: In recent years, the use of convolutional neural network (CNN) to classify images of midpalatal suture with different maturation stages has brought positive significance to the decision of the clinical maxillary expansion method. However, CNN cannot adequately learn the long-distance dependencies between images and features, which are also required for global recognition of midpalatal suture CBCT images. The Self-Attention of ViT has the function of capturing the relationship between long-distance pixels of the image. However, it lacks the inductive bias of CNN and needs more data training. To solve this problem, a CNN-enhanced ViT model based on transfer learning is proposed to classify midpalatal suture CBCT images. In this study, 2518 CBCT images of the palate plane are collected, and the images are divided into 1259 images as the training set, 506 images as the verification set, and 753 images as the test set. After the training set image preprocessing, the CNN-enhanced ViT model is trained and adjusted, and the generalization ability of the model is tested on the test set.

Results: The classification accuracy of our proposed ViT model is 95.75%, and its Macro-averaging Area under the receiver operating characteristic Curve (AUC) and Micro-averaging AUC are 97.89% and 98.36% respectively on our data test set. The classification accuracy of the best performing CNN model EfficientnetV2_S was 93.76% on our data test set. The classification accuracy of the clinician is 89.10% on our data test set.

Conclusions: The experimental results show that this method can effectively complete CBCT images classification of midpalatal suture maturation stages, and the performance is better than a clinician. Therefore, the model can provide a valuable reference for orthodontists and assist them in making correct a diagnosis.

Keywords: Cone beam computed tomography images; Midpalatal suture maturation stages; Self-attention; Vision transformer.

Fig. 1

Our data acquisition process

Fig. 2

(A) In stage A, the MPS presented a relatively straight, high-density white line with no or little curvature; (B) In stage B, there is a serrated white line of high-density in the MPS, and there may be two parallel high-density lines or low-density masses locally; (C) In stage C, two parallel serrated white lines of high-density appear in the midpalate suture, separated in some areas by small low-density masses; (D) In stage D, the MPS located in the posterior palatine bone has a similar density to the surrounding bone and is no longer visible. The MPS in the maxillary part has not yet fused and still presents two serrated high-density lines; (E) In stage E, the MPS of the maxilla is no longer visible, and the density is consistent with the surrounding bone

Fig. 3

Image fusion of curved palate plane

Fig. 4

Proposed ViT predicts the maturation stage of MPS: A The proposed ViT network structure; B Prediction process of MPS CBCT images maturation stage

Fig. 5

The architecture of the transformer encoder

Fig. 6

The architecture of the multi-head self-attention layer

Fig. 7

Training/validation loss and accuracy of the proposed model: (a) Loss curve; and (b) Accuracy curve

Fig. 8

The confusion matrix of indicators

Fig. 9

Confusion matrices of test results with and without balanced sampling on our dataset: (a) No balanced sampling; (b) Torchsampler balanced sampling; (c) Random undersampling; (d) Random oversampling

Fig. 10

The test accuracy curve of nine models on our test set: A Test accuracy curves of MobileNetV2, EfficientNet_b4, Vgg16 and our proposed ViT models; B Test accuracy curves of ResNet18, ResNet50, ResNet101, Inceptionv3, Efficientnetv2_s and our proposed ViT model

Fig. 11

The ROC curves and corresponding AUC values of nine models (A MobileNetV2; B ResNet50; C ResNet101; D ResNet18; E Inceptionv3, F Efficientnetv2_s; G EfficientNet_b4; H Vgg16; I Our proposed ViT model;) on our test set

Fig. 12

The original input images of the MPS maturation stage A to E and the corresponding CAM of the input images