Benign and Malignant Breast Tumor Classification in Ultrasound and Mammography Images via Fusion of Deep Learning and Handcraft Features

Abstract

Breast cancer is a disease that affects women in different countries around the world. The real cause of breast cancer is particularly challenging to determine, and early detection of the disease is necessary for reducing the death rate, due to the high risks associated with breast cancer. Treatment in the early period can increase the life expectancy and quality of life for women. CAD (Computer Aided Diagnostic) systems can perform the diagnosis of the benign and malignant lesions of breast cancer using technologies and tools based on image processing, helping specialist doctors to obtain a more precise point of view with fewer processes when making their diagnosis by giving a second opinion. This study presents a novel CAD system for automated breast cancer diagnosis. The proposed method consists of different stages. In the preprocessing stage, an image is segmented, and a mask of a lesion is obtained; during the next stage, the extraction of the deep learning features is performed by a CNN-specifically, DenseNet 201. Additionally, handcrafted features (Histogram of Oriented Gradients (HOG)-based, ULBP-based, perimeter area, area, eccentricity, and circularity) are obtained from an image. The designed hybrid system uses CNN architecture for extracting deep learning features, along with traditional methods which perform several handcraft features, following the medical properties of the disease with the purpose of later fusion via proposed statistical criteria. During the fusion stage, where deep learning and handcrafted features are analyzed, the genetic algorithms as well as mutual information selection algorithm, followed by several classifiers (XGBoost, AdaBoost, Multilayer perceptron (MLP)) based on stochastic measures, are applied to choose the most sensible information group among the features. In the experimental validation of two modalities of the CAD design, which performed two types of medical studies-mammography (MG) and ultrasound (US)-the databases mini-DDSM (Digital Database for Screening Mammography) and BUSI (Breast Ultrasound Images Dataset) were used. Novel CAD systems were evaluated and compared with recent state-of-the-art systems, demonstrating better performance in commonly used criteria, obtaining ACC of 97.6%, PRE of 98%, Recall of 98%, F1-Score of 98%, and IBA of 95% for the abovementioned datasets.

Keywords: breast cancer; feature selection; fusion; genetic algorithm; mammography image; mutual information; ultrasound image.

Figure 1

(a) Original US image of a benign lesion, (b) Benign lesion mask image found in BUSI dataset, (c) Original US image of a malignant lesion, (d) Malignant lesion mask image found in BUSI dataset.

Figure 2

(a) Original MG image of a benign lesion, (b) Benign lesion mask image found in mini-DDSM dataset, dilated with a filter ($7\times 7$) for better visualization, (c) Original MG image of a malignant lesion, (d) Malignant lesion mask image found in mini-DDSM dataset, dilated with a filter ($7\times 7$) for better visualization.

Figure 3

Block diagram of the proposed Computer Aided Diagnostic system.

Figure 4

(a) Original MG image obtained from mini_DDSM. (b) Region of interest (ROI) of a lesion in an MG image.

Figure 5

(a) ROI of a lesion obtained from MG image. (b) Mask of the lesion segmented from an ROI image.

Figure 6

DenseNET architecture.

Figure 7

Process of generation by LBP for the central pixel.

Figure 8

(a) ROI image obtained from MG image; (b) Generated LBP image of (a), where one can see the texture of the lesion.

Figure 9

Explanations of how 4 cells (in $2\times 2$) overlap the cells with a stride of 8 pixels together, forming a block.

Figure 10

(a) ROI image obtained from US image; (b) Generated HOG image of (a), where one can see the texture of the lesion.

Figure 11

Dependence of variances for PCA components in the selection of HOG and ULBP features.

Figure 12

Perceptual description of a mass based on the BI-RADS medical classification algorithm.

Figure 13

(a) Representation of an individual (gene); (b) Illustration of the crossover operation; (c) Mutation operation.

Figure 14

Features most representative from a US image, employing the genetic algorithm and mutual information algorithm fusion features.

Figure 15

Most representative features from an MG image, determined by employing the genetic algorithm and mutual information.

Figure 16

Most representative features from the fusion of US and MG images when employing the genetic and mutual information algorithms.

Figure 17

Confusion matrix when genetic and mutual information selection algorithms for features were employed for the compound (US + MG) dataset, (a) using XGBoost, (b) using MLP, and (c) using AdaBoost classifiers.