Early Diagnosis of Oral Squamous Cell Carcinoma Based on Histopathological Images Using Deep and Hybrid Learning Approaches

Abstract

Oral squamous cell carcinoma (OSCC) is one of the most common head and neck cancer types, which is ranked the seventh most common cancer. As OSCC is a histological tumor, histopathological images are the gold diagnosis standard. However, such diagnosis takes a long time and high-efficiency human experience due to tumor heterogeneity. Thus, artificial intelligence techniques help doctors and experts to make an accurate diagnosis. This study aimed to achieve satisfactory results for the early diagnosis of OSCC by applying hybrid techniques based on fused features. The first proposed method is based on a hybrid method of CNN models (AlexNet and ResNet-18) and the support vector machine (SVM) algorithm. This method achieved superior results in diagnosing the OSCC data set. The second proposed method is based on the hybrid features extracted by CNN models (AlexNet and ResNet-18) combined with the color, texture, and shape features extracted using the fuzzy color histogram (FCH), discrete wavelet transform (DWT), local binary pattern (LBP), and gray-level co-occurrence matrix (GLCM) algorithms. Because of the high dimensionality of the data set features, the principal component analysis (PCA) algorithm was applied to reduce the dimensionality and send it to the artificial neural network (ANN) algorithm to diagnose it with promising accuracy. All the proposed systems achieved superior results in histological image diagnosis of OSCC, the ANN network based on the hybrid features using AlexNet, DWT, LBP, FCH, and GLCM achieved an accuracy of 99.1%, specificity of 99.61%, sensitivity of 99.5%, precision of 99.71%, and AUC of 99.52%.

Keywords: ANN; CNN; DWT; FCH; GLCM; LBP; OSCC; SVM; hybrid method.

Figure 1

Structure of the histopathological image diagnostics methodology for early diagnosis of OSCC.

Figure 2

Samples of histopathological images of OSCC. (a) Original images before the enhancement process; (b) optimized images after the enhancement process.

Figure 3

Methodology for histopathological image diagnosis of oral cancer by a hybrid technique between CNN and SVM models.

Figure 4

Methodology for histopathological image diagnosis of oral cancer using the hybrid features technique between CNN model, DWT, LBP, FCH, and GLCM models.

Figure 5

Methodology for histopathological image diagnosis of oral cancer using ANN based on deep feature AlexNet and ResNet-18 models.

Figure 6

Evaluation of histopathological images for the diagnosis of OSCC.

Figure 7

Confusion matrix for performing hybrid techniques for OSCC data set diagnostics. (a) AlexNet + SVM; (b) ResNet-18 + SVM.

Figure 8

Error histogram for evaluating ANN performance based on hybrid features. (a) AlexNet, DWT, LBP, FCH, and GLCM; (b) ResNet-18, DWT, LBP, FCH, and GLCM.

Figure 9

Gradient for evaluating ANN performance based on hybrid features. (a) AlexNet, DWT, LBP, FCH, and GLCM; (b) ResNet-18, DWT, LBP, FCH, and GLCM.

Figure 10

ROC for evaluating ANN performance based on hybrid features. (a) AlexNet, DWT, LBP, FCH, and GLCM; (b) ResNet-18, DWT, LBP, FCH, and GLCM.

Figure 11

Best validation for evaluating ANN performance based on hybrid features. (a) AlexNet, DWT, LBP, FCH, and GLCM; (b) ResNet-18, DWT, LBP, FCH, and GLCM.

Figure 12

Confusion matrix for evaluating ANN performance based on hybrid features. (a) AlexNet, DWT, LBP, FCH, and GLCM; (b) ResNet-18, DWT, LBP, FCH, and GLCM.

Figure 13

ANN performance based on the hybrid features between CNN models and traditional algorithms.

Figure 14

Display of ANN performance based on the features of AlexNet and ResNet-18 models.

Figure 15

Confusion matrix for OSCC data set diagnosis by ANN based on deep features of models. (a) AlexNet; (b) ResNet-18.

Figure 16

Performance of the proposed methods for OSCC data set diagnostics.

Figure 17

Comparison of the performance of our system with previous studies for diagnosing oral squamous cell carcinomas [6,10,16,37,38,39].