Artificial Intelligence for Colonoscopy: Past, Present, and Future

Abstract

During the past decades, many automated image analysis methods have been developed for colonoscopy. Real-time implementation of the most promising methods during colonoscopy has been tested in clinical trials, including several recent multi-center studies. All trials have shown results that may contribute to prevention of colorectal cancer. We summarize the past and present development of colonoscopy video analysis methods, focusing on two categories of artificial intelligence (AI) technologies used in clinical trials. These are (1) analysis and feedback for improving colonoscopy quality and (2) detection of abnormalities. Our survey includes methods that use traditional machine learning algorithms on carefully designed hand-crafted features as well as recent deep-learning methods. Lastly, we present the gap between current state-of-the-art technology and desirable clinical features and conclude with future directions of endoscopic AI technology development that will bridge the current gap.

Fig. 1.

Diagram showing the colon anatomy.

Fig. 2.

Overview of the topics (in Sections II–V) summarized in this survey.

Fig. 3.

Examples (a) wall view; (b) lumen view; (c) spiral score and feedback; (d) retroflexion for viewing a difficult-to-reach area.

Fig. 4.

Examples of pixel-based interpretation on polyp images of the Kvasir V2 public dataset [17]: (a) input image, (b) gradient, (c) LRP, (d) Deep Taylor, and (e) Grad-CAM.

Fig. 5.

Examples of (a) ProtoPNet [187] and (b) two-level hierarchical concept-based interpretation [188] on polyp images of the Kvasir V2 public dataset [17]. The blue and yellow boxes indicate the image object that is used for prediction. Level 2 shows the object level concepts (e.g., the polyp object). Level 1 shows the low-level concepts (different shades of red colors and texture) that make up the polyp object. Thicker connecting lines indicate stronger influence of the lower-level to higher-level concepts.