Comparison of diagnostic performance between convolutional neural networks and human endoscopists for diagnosis of colorectal polyp: A systematic review and meta-analysis

Abstract

Prospective randomized trials and observational studies have revealed that early detection, classification, and removal of neoplastic colorectal polyp (CP) significantly improve the prevention of colorectal cancer (CRC). The current effectiveness of the diagnostic performance of colonoscopy remains unsatisfactory with unstable accuracy. The convolutional neural networks (CNN) system based on artificial intelligence (AI) technology has demonstrated its potential to help endoscopists in increasing diagnostic accuracy. Nonetheless, several limitations of the CNN system and controversies exist on whether it provides a better diagnostic performance compared to human endoscopists. Therefore, this study sought to address this issue. Online databases (PubMed, Web of Science, Cochrane Library, and EMBASE) were used to search for studies conducted up to April 2020. Besides, the quality assessment of diagnostic accuracy scale-2 (QUADAS-2) was used to evaluate the quality of the enrolled studies. Moreover, publication bias was determined using the Deeks' funnel plot. In total, 13 studies were enrolled for this meta-analysis (ranged between 2016 and 2020). Consequently, the CNN system had a satisfactory diagnostic performance in the field of CP detection (sensitivity: 0.848 [95% CI: 0.692-0.932]; specificity: 0.965 [95% CI: 0.946-0.977]; and AUC: 0.98 [95% CI: 0.96-0.99]) and CP classification (sensitivity: 0.943 [95% CI: 0.927-0.955]; specificity: 0.894 [95% CI: 0.631-0.977]; and AUC: 0.95 [95% CI: 0.93-0.97]). In comparison with human endoscopists, the CNN system was comparable to the expert but significantly better than the non-expert in the field of CP classification (CNN vs. expert: RDOR: 1.03, P = 0.9654; non-expert vs. expert: RDOR: 0.29, P = 0.0559; non-expert vs. CNN: 0.18, P = 0.0342). Therefore, the CNN system exhibited a satisfactory diagnostic performance for CP and could be used as a potential clinical diagnostic tool during colonoscopy.

Fig 1. Flow chart of studies identified, excluded and included.

Fig 2. Methodological quality of the included 13 studies using assessment tool of QUADAS-2.

(A) Grouped bar charts of risk of bias (left) and concerns for applicability (right). (B) Quality assessment for each individual study. QUADAS-2 = Quality Assessment of Diagnostic Accuracy Studies-2.

Fig 3. The pooled diagnostic accuracy index of CNN system in the field of CP detection.

(A) Sensitivity, (B) specificity. a: full videos; b:short videos. CNN: convolutional neural networks; CP: colorectal polyps. NBI: narrow-blue images; WLI: white-light images.

Fig 4. The pooled diagnostic accuracy index of CNN system in the field of CP classification.

(A) Sensitivity, (B) specificity. a: rectosigmoid; b: proximal-rectosigmoid. CNN: convolutional neural networks; CP: colorectal polyps.

Fig 5

Summary receiver operation characteristic (SROC) curve of diagnostic performance of CNN system (A), expert (B), and non-expert (C) for CP classification.

Fig 6. Deeks’ funnel plot for publication bias.

(A) CNN system for CP detection, (B) CNN system for CP classification.