RNA-Seq analysis for breast cancer detection: a study on paired tissue samples using hybrid optimization and deep learning techniques

Abstract

Problem: Breast cancer is a leading global health issue, contributing to high mortality rates among women. The challenge of early detection is exacerbated by the high dimensionality and complexity of gene expression data, which complicates the classification process.

Aim: This study aims to develop an advanced deep learning model that can accurately detect breast cancer using RNA-Seq gene expression data, while effectively addressing the challenges posed by the data's high dimensionality and complexity.

Methods: We introduce a novel hybrid gene selection approach that combines the Harris Hawk Optimization (HHO) and Whale Optimization (WO) algorithms with deep learning to improve feature selection and classification accuracy. The model's performance was compared to five conventional optimization algorithms integrated with deep learning: Genetic Algorithm (GA), Artificial Bee Colony (ABC), Cuckoo Search (CS), and Particle Swarm Optimization (PSO). RNA-Seq data was collected from 66 paired samples of normal and cancerous tissues from breast cancer patients at the Jawaharlal Nehru Cancer Hospital & Research Centre, Bhopal, India. Sequencing was performed by Biokart Genomics Lab, Bengaluru, India.

Results: The proposed model achieved a mean classification accuracy of 99.0%, consistently outperforming the GA, ABC, CS, and PSO methods. The dataset comprised 55 female breast cancer patients, including both early and advanced stages, along with age-matched healthy controls.

Conclusion: Our findings demonstrate that the hybrid gene selection approach using HHO and WO, combined with deep learning, is a powerful and accurate tool for breast cancer detection. This approach shows promise for early detection and could facilitate personalized treatment strategies, ultimately improving patient outcomes.

Keywords: Breast cancer; Deep learning; Harris Hawk algorithm; Whale optimization algorithm.

Fig. 1

Workflow of the steps followed in the study

Fig. 2

Depicts the systematic process employed for conducting a literature review

Fig. 3

Publication from the year 2015–2024 based on breast cancer with deep learning, machine learning

Algorithm 1

Fig. 4

Deep learning configuration

Fig. 5

The fitness graph of the proposed model

Fig. 6

The prediction graph of proposed model

Fig. 7

The energy value graph of the proposed algorithm

Fig. 8

Confusion matrix for a HHWO, b HHO, c HHO and d PSO

Fig. 9

Confusion matrix for a ABC, b GA, c CS and d SSA

Fig. 10

ROC for deep learning model optimized by HHWO, WOA, and HHO

Fig. 11

ROC curve of comparison algorithms

Fig. 12

ROC Curve of different classifiers including SVM, KNN, DT, and NB

Fig. 13

Comparison histogram based on Table 11