A review and comparative study of cancer detection using machine learning: SBERT and SimCSE application

Abstract

Background: Using visual, biological, and electronic health records data as the sole input source, pretrained convolutional neural networks and conventional machine learning methods have been heavily employed for the identification of various malignancies. Initially, a series of preprocessing steps and image segmentation steps are performed to extract region of interest features from noisy features. Then, the extracted features are applied to several machine learning and deep learning methods for the detection of cancer.

Methods: In this work, a review of all the methods that have been applied to develop machine learning algorithms that detect cancer is provided. With more than 100 types of cancer, this study only examines research on the four most common and prevalent cancers worldwide: lung, breast, prostate, and colorectal cancer. Next, by using state-of-the-art sentence transformers namely: SBERT (2019) and the unsupervised SimCSE (2021), this study proposes a new methodology for detecting cancer. This method requires raw DNA sequences of matched tumor/normal pair as the only input. The learnt DNA representations retrieved from SBERT and SimCSE will then be sent to machine learning algorithms (XGBoost, Random Forest, LightGBM, and CNNs) for classification. As far as we are aware, SBERT and SimCSE transformers have not been applied to represent DNA sequences in cancer detection settings.

Results: The XGBoost model, which had the highest overall accuracy of 73 ± 0.13 % using SBERT embeddings and 75 ± 0.12 % using SimCSE embeddings, was the best performing classifier. In light of these findings, it can be concluded that incorporating sentence representations from SimCSE's sentence transformer only marginally improved the performance of machine learning models.

Keywords: Cancer detection; DNA; Machine learning; SentenceBert; SimCSE.

Fig. 1

Generalized machine learning framework for lung cancer prediction [33]

Fig. 2

Generalized machine learning framework for breast cancer prediction [45]

Fig. 3

Generalized machine learning framework for prostate cancer prediction using 3-d CNNs, pooling layers, and a fully connected layer for classification [69]

Fig. 4

Using a deep CNN network to predict colorectal cancer outcome using images [86]

Fig. 5

SBERT architecture with classification objective function (left) and the regression objective function (right) [105]

Fig. 6

Unsupervised SimCSE (a) and supervised SimCSE (b) [110]

Fig. 7

Visualisation of the SBERT documents with k-means clustering

Fig. 8

Visualisation of the SimCSE documents with k-means clustering

Fig. 9

Confusion matrix of the LightGBM model using SBERT representations after SMOTE (dev set)

Fig. 10

Confusion matrix of the XGBoost model using SBERT representations after SMOTE (dev set)

Fig. 11

Confusion matrix of the LightGBM model using SimCSE representations after SMOTE (dev set)

Fig. 12

Confusion matrix of the Random forest model using SimCSE representations after SMOTE (dev set)