Hybrid Models Based on Fusion Features of a CNN and Handcrafted Features for Accurate Histopathological Image Analysis for Diagnosing Malignant Lymphomas

Abstract

Malignant lymphoma is one of the most severe types of disease that leads to death as a result of exposure of lymphocytes to malignant tumors. The transformation of cells from indolent B-cell lymphoma to B-cell lymphoma (DBCL) is life-threatening. Biopsies taken from the patient are the gold standard for lymphoma analysis. Glass slides under a microscope are converted into whole slide images (WSI) to be analyzed by AI techniques through biomedical image processing. Because of the multiplicity of types of malignant lymphomas, manual diagnosis by pathologists is difficult, tedious, and subject to disagreement among physicians. The importance of artificial intelligence (AI) in the early diagnosis of malignant lymphoma is significant and has revolutionized the field of oncology. The use of AI in the early diagnosis of malignant lymphoma offers numerous benefits, including improved accuracy, faster diagnosis, and risk stratification. This study developed several strategies based on hybrid systems to analyze histopathological images of malignant lymphomas. For all proposed models, the images and extraction of malignant lymphocytes were optimized by the gradient vector flow (GVF) algorithm. The first strategy for diagnosing malignant lymphoma images relied on a hybrid system between three types of deep learning (DL) networks, XGBoost algorithms, and decision tree (DT) algorithms based on the GVF algorithm. The second strategy for diagnosing malignant lymphoma images was based on fusing the features of the MobileNet-VGG16, VGG16-AlexNet, and MobileNet-AlexNet models and classifying them by XGBoost and DT algorithms based on the ant colony optimization (ACO) algorithm. The color, shape, and texture features, which are called handcrafted features, were extracted by four traditional feature extraction algorithms. Because of the similarity in the biological characteristics of early-stage malignant lymphomas, the features of the fused MobileNet-VGG16, VGG16-AlexNet, and MobileNet-AlexNet models were combined with the handcrafted features and classified by the XGBoost and DT algorithms based on the ACO algorithm. We concluded that the performance of the two networks XGBoost and DT, with fused features between DL networks and handcrafted, achieved the best performance. The XGBoost network based on the fused features of MobileNet-VGG16 and handcrafted features resulted in an AUC of 99.43%, accuracy of 99.8%, precision of 99.77%, sensitivity of 99.7%, and specificity of 99.8%. This highlights the significant role of AI in the early diagnosis of malignant lymphoma, offering improved accuracy, expedited diagnosis, and enhanced risk stratification. This study highlights leveraging AI techniques and biomedical image processing; the analysis of whole slide images (WSI) converted from biopsies allows for improved accuracy, faster diagnosis, and risk stratification. The developed strategies based on hybrid systems, combining deep learning networks, XGBoost and decision tree algorithms, demonstrated promising results in diagnosing malignant lymphoma images. Furthermore, the fusion of handcrafted features with features extracted from DL networks enhanced the performance of the classification models.

Keywords: ACO; DT; GVF; XGBoost; deep learning; fusion features; malignant lymphoma.

Figure 1

Random samples from all classes of the malignant lymphoma dataset: (a). original images, (b). improved images.

Figure 2

WSI images from all classes of the malignant lymphoma dataset after application of the GVF algorithm: (a). original images; (b). after segmentation; (c). malignant lymphoma cells (ROI).

Figure 3

Analysis of WSI images for the diagnosis of malignant lymphomas by the XGBoost and DT networks based on the features of DL models.

Figure 4

Analysis of WSI images for the diagnosis of malignant lymphomas by the XGBoost and DT networks based on the fused features of DL models.

Figure 5

Analysis of WSI images for the diagnosis of malignant lymphomas by the XGBoost and DT networks based on fused features between DL models and handcrafted features.

Figure 6

Confusion matrix showing performance results of DL-XGBoost hybrid models based on the ACO method for the diagnosis of malignant lymphomas: (a). MobileNet-XGBoost; (b). VGG16-XGBoost; (c). AlexNet-XGBoost.

Figure 7

Confusion matrix showing performance results of DL-DT hybrid models based on the ACO method for the diagnosis of malignant lymphomas. (a). MobileNet-DT, (b). VGG16-DT, (c). AlexNet-DT.

Figure 8

Confusion matrix showing performance results of XGBoost with hybrid features of DL models based on the ACO method for diagnosing malignant lymphomas. (a). MobileNet-VGG16-XGBoost, (b). VGG16-XAlexNet-XGBoost, (c). MobileNet-AlexNet-XGBoost.

Figure 9

Confusion matrix showing performance results of DT with hybrid features of DL models based on the ACO method for diagnosing malignant lymphomas. (a). MobileNet-VGG16-DT, (b). VGG16-XAlexNet-DT, (c). MobileNet-AlexNet-DT.

Figure 10

Confusion matrix showing performance results of XGBoost with hybrid features of DL and handcrafted features based on the ACO method for diagnosing malignant lymphomas. (a). MobileNet-VGG16-Handcrafted-XGBoost, (b). VGG16-XAlexNet-Handcrafted-XGBoost, (c). MobileNet-AlexNet-Handcrafted-XGBoost.

Figure 11

Confusion matrix showing performance results of DT with hybrid features of DL and handcrafted features based on the ACO method for diagnosing malignant lymphomas. (a). MobileNet-VGG16-Handcrafted-DT, (b). VGG16-AlexNet-Handcrafted-DT, (c). MobileNet-AlexNet-Handcrafted-DT.