Intraoperative near infrared functional imaging of rectal cancer using artificial intelligence methods - now and near future state of the art

Abstract

Colorectal cancer remains a major cause of cancer death and morbidity worldwide. Surgery is a major treatment modality for primary and, increasingly, secondary curative therapy. However, with more patients being diagnosed with early stage and premalignant disease manifesting as large polyps, greater accuracy in diagnostic and therapeutic precision is needed right from the time of first endoscopic encounter. Rapid advancements in the field of artificial intelligence (AI), coupled with widespread availability of near infrared imaging (currently based around indocyanine green (ICG)) can enable colonoscopic tissue classification and prognostic stratification for significant polyps, in a similar manner to contemporary dynamic radiological perfusion imaging but with the advantage of being able to do so directly within interventional procedural time frames. It can provide an explainable method for immediate digital biopsies that could guide or even replace traditional forceps biopsies and provide guidance re margins (both areas where current practice is only approximately 80% accurate prior to definitive excision). Here, we discuss the concept and practice of AI enhanced ICG perfusion analysis for rectal cancer surgery while highlighting recent and essential near-future advancements. These include breakthrough developments in computer vision and time series analysis that allow for real-time quantification and classification of fluorescent perfusion signals of rectal cancer tissue intraoperatively that accurately distinguish between normal, benign, and malignant tissues in situ endoscopically, which are now undergoing international prospective validation (the Horizon Europe CLASSICA study). Next stage advancements may include detailed digital characterisation of small rectal malignancy based on intraoperative assessment of specific intratumoral fluorescent signal pattern. This could include T staging and intratumoral molecular process profiling (e.g. regarding angiogenesis, differentiation, inflammatory component, and tumour to stroma ratio) with the potential to accurately predict the microscopic local response to nonsurgical treatment enabling personalised therapy via decision support tools. Such advancements are also applicable to the next generation fluorophores and imaging agents currently emerging from clinical trials. In addition, by providing an understandable, applicable method for detailed tissue characterisation visually, such technology paves the way for acceptance of other AI methodology during surgery including, potentially, deep learning methods based on whole screen/video detailing.

Keywords: Artificial intelligence; Clinical trials; Digital surgery; Dynamic imaging; Fluorescence guided surgery (FGS); Indocyanine green; Intraoperative imaging; Rectal cancer.

Fig. 1

Schematic showing time series changes in ICG after systemic administration in both normal tissue versus a malignant focus in rectal tissue. Among other factors, cancer angiogenesis results in distorted tissue architecture including increased capillary volume and permeability which leads to increased ICG leakage and decreased clearance versus what happens in normal and purely benign lesions. This results in retention of ICG within the malignant core of the polyp when levels have otherwise significantly decreased in normal and benign tissues

Fig. 2

Indicative Progression of ICG fluorescence in a malignant lesion (red border) and health control (green) with a corresponding time series intensity quantification taken from a representative region within the abnormal area compared to a region from the adjacent healthy tissue (interestingly this lesion was initially judged benign by the endoscopist based on clinical impression). Fluorescent appearances are quickly distinctive for malignancy which was confirmed on pathology after its excision. Image A shows the subtle area of abnormality seen endoscopically. Image B at 7 s from ICG dosage shows nuanced lack of uptake in the tumour relative to the surrounding healthy tissue. At approximately 30 s image C shows that the fluorescence of the malignant tissue has now matched or exceeded that of the healthy tissue. Image D at 3 min shows significant retention of dye relative to the washout of the healthy tissue. These same trends are more evident in the time series graph (image E) demonstrating delayed ICG entry and clearance in the cancerous tissue. Curve features for each case are extracted and serve as the data inputted into machine learning classification pipeline which produces a prediction of “cancer” or “benign”

Fig. 3

Frozen tissue samples of colorectal cancer (A) 15 min and (B & C) 2 h following systemic indocyanine green injection. Hematoxylin and Eosin stained (left) and microscopic fluorescence imaging of unstained tissue (right) demonstrate a predominance of ICG within stromal tissue and vasculature and a relative lack of ICG within neoplastic malignant glands (circled) at the early timepoint (A) Trapping of ICG within the benign: malignant tissue interphase is seen at the later time points B & C. ICG is also noted to sporadically trap within non-malignant tissues adjacent to cancer, likely as a result of distorted architecture from the nearby malignant process (similar levels of distortion can also be seen in non-malignant inflammatory tissues resulting in false positive appearances associated with single point-in-time ICG assessment). A Nikon Eclipse Ti2 Inverted Research Microscope and a LI-COR Odyssey DLx Near-Infrared Fluorescence Imaging System were used to analyse obtained samples. Specimens were mounted in OCT, flash frozen using Lamb’s freezing aerosol and cut using a cryotome to 5 micrometre thick levels

Fig. 4

Schematic showing the end-to-end AI computing method proposed following endoscopic lesion classification by a surgeon or gastroenterologist. Image A shows the annotated areas with the suspicious lesion within the red border and healthy control outlined by green. Images B and C show the first and last frames respectively with tracked regions. With image stabilisation, analytical grids are applied to the image and analysed for ICG inflow by evidence of their fluorescence intensity, producing corresponding curves. Trained mathematical models are deployed to turn the curve features into a formula that outputs a prediction regarding cancer likelihood in comparison to that of an area without disease visible on the same screen at the same time