Ensemble Method of Convolutional Neural Networks with Directed Acyclic Graph Using Dermoscopic Images: Melanoma Detection Application

Abstract

The early detection of melanoma is the most efficient way to reduce its mortality rate. Dermatologists achieve this task with the help of dermoscopy, a non-invasive tool allowing the visualization of patterns of skin lesions. Computer-aided diagnosis (CAD) systems developed on dermoscopic images are needed to assist dermatologists. These systems rely mainly on multiclass classification approaches. However, the multiclass classification of skin lesions by an automated system remains a challenging task. Decomposing a multiclass problem into a binary problem can reduce the complexity of the initial problem and increase the overall performance. This paper proposes a CAD system to classify dermoscopic images into three diagnosis classes: melanoma, nevi, and seborrheic keratosis. We introduce a novel ensemble scheme of convolutional neural networks (CNNs), inspired by decomposition and ensemble methods, to improve the performance of the CAD system. Unlike conventional ensemble methods, we use a directed acyclic graph to aggregate binary CNNs for the melanoma detection task. On the ISIC 2018 public dataset, our method achieves the best balanced accuracy (76.6%) among multiclass CNNs, an ensemble of multiclass CNNs with classical aggregation methods, and other related works. Our results reveal that the directed acyclic graph is a meaningful approach to develop a reliable and robust automated diagnosis system for the multiclass classification of dermoscopic images.

Keywords: computer-aided system; deep learning; dermoscopic images; directed acyclic graph; ensemble method; fusion-based model; melanoma detection; multiclass classification; skin cancer.

Figure 1

Pigmented skin lesions taken from ISIC [5] dataset. Typical lesions (a) melanoma, (b) nevi, and (c) seborrheic keratosis do not pose any diagnosis issues for dermatologists, whereas atypical lesions (d) melanoma, (e) nevi, and (f) seborrheic keratosis, having poor intra-lesion features, are much more challenging to differentiate.

Figure 2

Photograph of a handheld dermoscopic sensor with non-polarized and polarized light, which is used by dermatologists during their clinical examination. The dermatoscope shown in the figure is produced by Dermoscope DermLite DL4, 3GEN Inc., San Juan Capistrano, CA, USA.

Figure 3

Block diagram of the proposed computer-aided diagnosis system. Skin images are first preprocessed. Then, three binary CNNs are trained using a one-versus-one approach to differentiate lesion i from another lesion j. Finally, the output of each CNN is aggregated using the directed acyclic graph (DAG) to output the final prediction.

Figure 4

A sample of images in the dataset before and after preprocessing.

Figure 5

An illustration of a plain block (left) and a residual block (right).

Figure 6

The structures of the convolutional networks used in our method. (top) The modified ResNet50, (middle) the modified VGG16, and (bottom) the modified VGG19. In both cases, we replaced the last fully connected (FC) layer with an FC layer with 2 nodes. MP and FL represent the max pooling and flattening layers, respectively.

Figure 7

The decision directed acyclic graph (DDAG) for finding the best of three class.

Figure 8

Receiver operating characteristic (ROC) curves of our best model. The area under the curve (AUC) of the ROC is provided for each lesion class: melanoma (MEL), seborrheic keratosis (SEK), and nevi (NEV).

Figure 9

Output of our computer-aided diagnosis for melanoma detection. Heatmap generation was implemented with Grad-CAM [40]. MEL: melanoma, SEK: seborrheic keratosis, NEV: nevi.