Feedforward Artificial Neural Network-Based Colorectal Cancer Detection Using Hyperspectral Imaging: A Step towards Automatic Optical Biopsy

Abstract

Currently, colorectal cancer (CRC) is mainly identified via a visual assessment during colonoscopy, increasingly used artificial intelligence algorithms, or surgery. Subsequently, CRC is confirmed through a histopathological examination by a pathologist. Hyperspectral imaging (HSI), a non-invasive optical imaging technology, has shown promising results in the medical field. In the current study, we combined HSI with several artificial intelligence algorithms to discriminate CRC. Between July 2019 and May 2020, 54 consecutive patients undergoing colorectal resections for CRC were included. The tumor was imaged from the mucosal side with a hyperspectral camera. The image annotations were classified into three groups (cancer, CA; adenomatous margin around the central tumor, AD; and healthy mucosa, HM). Classification and visualization were performed based on a four-layer perceptron neural network. Based on a neural network, the classification of CA or AD resulted in a sensitivity of 86% and a specificity of 95%, by means of leave-one-patient-out cross-validation. Additionally, significant differences in terms of perfusion parameters (e.g., oxygen saturation) related to tumor staging and neoadjuvant therapy were observed. Hyperspectral imaging combined with automatic classification can be used to differentiate between CRC and healthy mucosa. Additionally, the biological changes induced by chemotherapy to the tissue are detectable with HSI.

Keywords: colorectal cancer (CRC); deep learning; hyperspectral imaging (HSI); machine learning; optical biopsy; optical imaging.

Figure 1

Annotation process. A pT3 adenocarcinoma of the sigmoid colon. The first annotation in yellow marks the certain tumor tissue; in red the surrounding tissue, probably with adenoma parts; in blue the healthy mucosa; and in black the deleted unmarked areas.

Figure 2

Averaged receiver operating characteristic (ROC) curve for every patient’s individual evaluation for classification: Cancer (CA) with adenomatous margin around the central tumor (AD) against healthy mucosa (HM), CA without AD against HM, and CA against AD (AUC—area under the curve).

Figure 3

(A,C,E,G) represent annotation and classification of cancerous tissue: A = adenoma; B = ypT2 pN1a (1/23) pM1a (PUL) L1 V0 Pn0, UICC Stage IV A; C = pT2 pN0 (0/18) M0 L0 V0 Pn0, UICC Stage I; D = pT3a pN0 (0/29) M0 L1 V0 Pn0, UICC Stage IIA; E = ypT3 ypN1b (2/21) M0 L1 V0 Pn0, UICC Stage yp III A. (red fill: cancer and adenoma, red line: cancer, green line: adenoma, blue fill: healthy mucosa, blue line: healthy mucosa).

Figure 4

Comparison between physiological parameters TWI, OHI, NIR-PI, and StO₂. (A) Comparison of healthy and cancerous tissue without neoadjuvant therapy. (B) Comparison of healthy and cancerous tissue with neoadjuvant therapy. (C) Comparison of cancerous tissue: (y)pT1 and (y)pT2 stage vs. (y)pT3 and (y)pT4 stage. (D) Comparison of cancerous tissue: ypT1 and ypT2 stage vs. ypT3 and ypT4 stage with neoadjuvant therapy.

Figure 5

An example of tissue specimen with a tumor (ypT2). Tissue structures show homogeneous physiological parameter values (blue: healthy mucosa, yellow: cancer, red: tumor margin).