AI Techniques of Dermoscopy Image Analysis for the Early Detection of Skin Lesions Based on Combined CNN Features

Abstract

Melanoma is one of the deadliest types of skin cancer that leads to death if not diagnosed early. Many skin lesions are similar in the early stages, which causes an inaccurate diagnosis. Accurate diagnosis of the types of skin lesions helps dermatologists save patients' lives. In this paper, we propose hybrid systems based on the advantages of fused CNN models. CNN models receive dermoscopy images of the ISIC 2019 dataset after segmenting the area of lesions and isolating them from healthy skin through the Geometric Active Contour (GAC) algorithm. Artificial neural network (ANN) and Random Forest (Rf) receive fused CNN features and classify them with high accuracy. The first methodology involved analyzing the area of skin lesions and diagnosing their type early using the hybrid models CNN-ANN and CNN-RF. CNN models (AlexNet, GoogLeNet and VGG16) receive lesions area only and produce high depth feature maps. Thus, the deep feature maps were reduced by the PCA and then classified by ANN and RF networks. The second methodology involved analyzing the area of skin lesions and diagnosing their type early using the hybrid CNN-ANN and CNN-RF models based on the features of the fused CNN models. It is worth noting that the features of the CNN models were serially integrated after reducing their high dimensions by Principal Component Analysis (PCA). Hybrid models based on fused CNN features achieved promising results for diagnosing dermatoscopic images of the ISIC 2019 data set and distinguishing skin cancer from other skin lesions. The AlexNet-GoogLeNet-VGG16-ANN hybrid model achieved an AUC of 94.41%, sensitivity of 88.90%, accuracy of 96.10%, precision of 88.69%, and specificity of 99.44%.

Keywords: ANN; PCA; RF; deep learning; fusion features; skin lesion.

Figure 1

Sample dermatoscopy images of the ISIC 2019 dataset (a) before improvement (b) after improvement.

Figure 2

A sample from each class of the ISIC 2019 dataset for extracting the lesion area and its isolation from the healthy skin using the GAC method (a) Original images (b) Enhanced images (c) Selection of the lesion area (d) Segmentation of the lesion area (e) Lesion area (ROI).

Figure 3

ANN structure for analyzing dermatoscopic images to diagnose the skin lesions of the ISIC 2019 dataset.

Figure 4

A hybrid model of CNN and machine learning for analyzing dermatoscopic images to diagnose the skin lesions of the ISIC 2019 dataset.

Figure 5

A hybrid model of ANN and RF with fused features of CNN for analyzing dermatoscopic images to diagnose the skin lesions of the ISIC 2019 dataset.

Figure 6

Display the number of dermoscopy images of the ISIC 2019 data set before and after data augmentation.

Figure 7

Display performance results of pre-trained CNN models for analysis of the ISIC 2019 dataset image for early diagnosis and distinction of skin cancer and other skin lesions.

Figure 8

Display Performance results of CNN models based on ROI using the GAC method for early diagnosis and distinction of skin cancer and other skin lesions.

Figure 9

Display Performance results of CNN-ANN hybrid models based on ROI using the GAC method for early diagnosis and distinction of skin cancer and other skin lesions.

Figure 10

Display Performance results of CNN-RF hybrid models based on ROI using the GAC method for early diagnosis and discrimination of skin cancer and other skin lesions.

Figure 11

Confusion matrix for displaying performance results of hybrid CNN-ANN models based on ROI using the GAC method for early diagnosis and discrimination of skin cancer from other skin lesions (a) AlexNet-ANN (b) GoogLeNet-ANN (c) VGG16-ANN.

Figure 12

Confusion matrix for displaying performance results of hybrid CNN-RF models based on ROI using the GAC method for early diagnosis and discrimination of skin cancer from other skin lesions (a) AlexNet-RF (b) GoogLeNet-RF (c) VGG16-RF.

Figure 13

Display Performance results of CNN-ANN hybrid models based on the fused CNN models for early diagnosis and discrimination of skin cancer and other skin lesions.

Figure 14

Display Performance results of CNN-RF hybrid models based on the fused CNN models for early diagnosis and discrimination of skin cancer and other skin lesions.

Figure 15

Confusion matrix for displaying performance results of CNN-ANN hybrid models based on the fused CNN models for early diagnosis and discrimination of skin cancer and other skin lesions (a) AlexNet-GoogLeNet-ANN (b) GoogLeNet-VGG16-ANN (c) AlexNet-VGG16-ANN (d) AlexNet-GoogLeNet-VGG16-ANN.

Figure 16

Confusion matrix for displaying performance results of CNN-RF hybrid models based on the fused CNN models for early diagnosis and discrimination of skin cancer and other skin lesions (a) AlexNet-GoogLeNet-RF (b) GoogLeNet-VGG16-RF (c) AlexNet-VGG16-RF (d) AlexNet-GoogLeNet-VGG16-RF.

Figure 17

Performance of the proposed systems for dermatoscopic image analysis for early diagnosis of skin cancer for the ISIC 2019 data set and distinguishing them from other skin lesions.