Multi-Method Analysis of Histopathological Image for Early Diagnosis of Oral Squamous Cell Carcinoma Using Deep Learning and Hybrid Techniques

Abstract

Oral cancer is a fatal disease and ranks seventh among the most common cancers throughout the whole globe. Oral cancer is a type of cancer that usually affects the head and neck. The current gold standard for diagnosis is histopathological investigation, however, the conventional approach is time-consuming and requires professional interpretation. Therefore, early diagnosis of Oral Squamous Cell Carcinoma (OSCC) is crucial for successful therapy, reducing the risk of mortality and morbidity, while improving the patient's chances of survival. Thus, we employed several artificial intelligence techniques to aid clinicians or physicians, thereby significantly reducing the workload of pathologists. This study aimed to develop hybrid methodologies based on fused features to generate better results for early diagnosis of OSCC. This study employed three different strategies, each using five distinct models. The first strategy is transfer learning using the Xception, Inceptionv3, InceptionResNetV2, NASNetLarge, and DenseNet201 models. The second strategy involves using a pre-trained art of CNN for feature extraction coupled with a Support Vector Machine (SVM) for classification. In particular, features were extracted using various pre-trained models, namely Xception, Inceptionv3, InceptionResNetV2, NASNetLarge, and DenseNet201, and were subsequently applied to the SVM algorithm to evaluate the classification accuracy. The final strategy employs a cutting-edge hybrid feature fusion technique, utilizing an art-of-CNN model to extract the deep features of the aforementioned models. These deep features underwent dimensionality reduction through principal component analysis (PCA). Subsequently, low-dimensionality features are combined with shape, color, and texture features extracted using a gray-level co-occurrence matrix (GLCM), Histogram of Oriented Gradient (HOG), and Local Binary Pattern (LBP) methods. Hybrid feature fusion was incorporated into the SVM to enhance the classification performance. The proposed system achieved promising results for rapid diagnosis of OSCC using histological images. The accuracy, precision, sensitivity, specificity, F-1 score, and area under the curve (AUC) of the support vector machine (SVM) algorithm based on the hybrid feature fusion of DenseNet201 with GLCM, HOG, and LBP features were 97.00%, 96.77%, 90.90%, 98.92%, 93.74%, and 96.80%, respectively.

Keywords: GLCM; HOG; LBP; PCA; SVM; hybrid model; oral squamous cell carcinoma (OSCC).

Figure 1

Proposed Methodology of OSCC.

Figure 2

Samples of histopathological images (a) Normal and (b) OSCC samples.

Figure 3

Deep learning architectures (a) Xception model (b) InceptionV3 model (c) InceptionResNetV2 model (d) NASNetLarge model, and (e) DenseNet201 model.

Figure 3

Deep learning architectures (a) Xception model (b) InceptionV3 model (c) InceptionResNetV2 model (d) NASNetLarge model, and (e) DenseNet201 model.

Figure 4

CNN and SVM hybrid models (a) Hybrid model between Xception and SVM (b) Hybrid model between InceptionV3 and SVM (c) Hybrid model between InceptionResNetV3 and SVM (d) Hybrid model between NASNetLarge and SVM, and (e) Hybrid model between DensNet201 and SVM.

Figure 4

CNN and SVM hybrid models (a) Hybrid model between Xception and SVM (b) Hybrid model between InceptionV3 and SVM (c) Hybrid model between InceptionResNetV3 and SVM (d) Hybrid model between NASNetLarge and SVM, and (e) Hybrid model between DensNet201 and SVM.

Figure 5

Hybrid features between CNN model, with HOG, GLCM, and LBP (a) Hybrid Features between Xception with HOG, GLCM, and LBP (b) Hybrid features between InceptionV3 with HOG, GLCM, and LBP (c) Hybrid features between InceptionResNetV2 with HOG, GLCM, and LBP (d) Hybrid features between NASNetLarge with HOG, GLCM, and LBP, and (e) Hybrid features between NASNetLarge with HOG, GLCM, and LBP.

Figure 5

Hybrid features between CNN model, with HOG, GLCM, and LBP (a) Hybrid Features between Xception with HOG, GLCM, and LBP (b) Hybrid features between InceptionV3 with HOG, GLCM, and LBP (c) Hybrid features between InceptionResNetV2 with HOG, GLCM, and LBP (d) Hybrid features between NASNetLarge with HOG, GLCM, and LBP, and (e) Hybrid features between NASNetLarge with HOG, GLCM, and LBP.

Figure 6

Xception model. (a) Training and validation accuracy (b) Training and validation loss (c) Confusion matrix.

Figure 7

InceptionV2. (a) Training and validation accuracy (b) Training and validation loss (c) Confusion matrix.

Figure 8

InceptionResNetV2. (a) Training and validation accuracy (b) Training and validation loss (c) Confusion matrix.

Figure 9

NASNetLarge. (a) Training and validation accuracy (b) Training and validation loss (c) Confusion matrix.

Figure 10

DenseNet201. (a) Training and validation accuracy (b) Training and validation loss (c) Confusion matrix.

Figure 11

CNN and SVM hybrid models confusion matrices. (a) Xception + SVM (b) InceptionV3 + SVM (c) InceptionResNetV2 + SVM (d) NASNetLarge + SVM and (e) DenseNet201 + SVM.

Figure 12

Deep Feature with (GLCM, HOG, and LBP) confusion matrices. (a) Xception + (GLCM, HOG, and LBP) + SVM (b) InceptionV3 + (GLCM, HOG, and LBP) + SVM (c) InceptionResNetV2 + (GLCM, HOG, and LBP) + SVM (d) NASNetLarge + (GLCM, HOG, and LBP) + SVM (e) DenseNet201 + (GLCM, HOG, and LBP) + SVM.

Figure 13

ROC of histopathological images of OSCC (a) Deep learning models (b) CNN with SVM (c) Hybrid Deep Feature with GLCM, HOG, and LBP.

Figure 14

Comparison of various models in terms of accuracy.

Figure 15

Comparison of Proposed Model with the existing model [17,18,21,32] in terms of accuracy, precision, specificity, and F1 score.