Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 May;36(5):3592-3600.
doi: 10.1007/s00464-021-08685-7. Epub 2021 Oct 12.

The use of deep learning on endoscopic images to assess the response of rectal cancer after chemoradiation

Hester E Haak #  1   2 , Xinpei Gao #  3 , Monique Maas  3 , Selam Waktola  3 , Sean Benson  3 , Regina G H Beets-Tan  2   3 , Geerard L Beets  1   2 , Monique van Leerdam  4 , Jarno Melenhorst  5   6
Affiliations

The use of deep learning on endoscopic images to assess the response of rectal cancer after chemoradiation

Hester E Haak et al. Surg Endosc. 2022 May.

Abstract

Background: Accurate response evaluation is necessary to select complete responders (CRs) for a watch-and-wait approach. Deep learning may aid in this process, but so far has never been evaluated for this purpose. The aim was to evaluate the accuracy to assess response with deep learning methods based on endoscopic images in rectal cancer patients after neoadjuvant therapy.

Methods: Rectal cancer patients diagnosed between January 2012 and December 2015 and treated with neoadjuvant (chemo)radiotherapy were retrospectively selected from a single institute. All patients underwent flexible endoscopy for response evaluation. Diagnostic performance (accuracy, area under the receiver operator characteristics curve (AUC), positive- and negative predictive values, sensitivities and specificities) of different open accessible deep learning networks was calculated. Reference standard was histology after surgery, or long-term outcome (>2 years of follow-up) in a watch-and-wait policy.

Results: 226 patients were included for the study (117(52%) were non-CRs; 109(48%) were CRs). The accuracy, AUC, positive- and negative predictive values, sensitivity and specificity of the different models varied from 0.67-0.75%, 0.76-0.83%, 67-74%, 70-78%, 68-79% to 66-75%, respectively. Overall, EfficientNet-B2 was the most successful model with the highest diagnostic performance.

Conclusions: This pilot study shows that deep learning has a modest accuracy (AUCs 0.76-0.83). This is not accurate enough for clinical decision making, and lower than what is generally reported by experienced endoscopists. Deep learning models can however be further improved and may become useful to assist endoscopists in evaluating the response. More well-designed prospective studies are required.

Keywords: Artificial intelligence; Deep learning; Organ preservation; Rectal cancer; Response evaluation; Watch-and-wait approach.

PubMed Disclaimer

Conflict of interest statement

Hester E. Haak, Selam Waktola, Monique Maas, Sean Benson, Xinpei Gao, Regina G.H. Beets-Tan, Geerard L. Beets, Monique van Leerdam and Jarno Melenhorst have no conflicts of interest or financial ties to disclose.

Figures

Fig. 1
Fig. 1
Example of evident and doubtful complete responders and non-complete responders. Evident complete response with a typical white scar (yellow arrows) (a), doubtful response with a small ulcer (yellow arrows) (b), doubtful response with a small-medium sized ulcer (yellow arrows) (c), and evident incomplete response with a tumor mass (d) (Color figure online)
Fig. 2
Fig. 2
Overview of the combined model architecture. [1408] Represents the last channels in EfficientNet-B2. [500] Represents the number of neurons in feedforward neural network based on three selected clinical features
Fig. 3
Fig. 3
ROC curve of EfficientNet-B2 for the endoscopic image model and combined model and ROC curve of feedforward neural network model for selected clinical variables. AUC Area under the ROC curve
See this image and copyright information in PMC

References

    1. Maas M, Lambregts DM, Nelemans PJ, Heijnen LA, Martens MH, Leijtens JW, et al. Assessment of clinical complete response after chemoradiation for rectal cancer with digital rectal examination, endoscopy, and MRI: selection for organ-saving treatment. Ann Surg Oncol. 2015;22(12):3873–3880. doi: 10.1245/s10434-015-4687-9. - DOI - PMC - PubMed
    1. van der Sande ME, Maas M, Melenhorst J, Breukink SO, van Leerdam ME, and Beets GL (2019) Predictive value of endoscopic features for a complete response after chemoradiotherapy for rectal cancer. Ann Surg. - PubMed
    1. Nahas SC, Rizkallah Nahas CS, Sparapan Marques CF, Ribeiro U, Jr, Cotti GC, Imperiale AR, et al. Pathologic complete response in rectal cancer: can we detect it? lessons learned from a proposed randomized trial of watch-and-wait treatment of rectal cancer. Dis Colon Rectum. 2016;59(4):255–263. doi: 10.1097/DCR.0000000000000558. - DOI - PubMed
    1. Ferlitsch M, Moss A, Hassan C, Bhandari P, Dumonceau JM, Paspatis G, et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) clinical guideline. Endoscopy. 2017;49(3):270–297. doi: 10.1055/s-0043-102569. - DOI - PubMed
    1. van der Sommen F, Curvers WL, Nagengast WB. Novel Developments in endoscopic mucosal imaging. Gastroenterology. 2018;154(7):1876–1886. doi: 10.1053/j.gastro.2018.01.070. - DOI - PubMed

LinkOut - more resources