A new artificial intelligence system successfully detects and localises early neoplasia in Barrett's esophagus by using convolutional neural networks

Abstract

Background and aims: Seattle protocol biopsies for Barrett's Esophagus (BE) surveillance are labour intensive with low compliance. Dysplasia detection rates vary, leading to missed lesions. This can potentially be offset with computer aided detection. We have developed convolutional neural networks (CNNs) to identify areas of dysplasia and where to target biopsy.

Methods: 119 Videos were collected in high-definition white light and optical chromoendoscopy with i-scan (Pentax Hoya, Japan) imaging in patients with dysplastic and non-dysplastic BE (NDBE). We trained an indirectly supervised CNN to classify images as dysplastic/non-dysplastic using whole video annotations to minimise selection bias and maximise accuracy. The CNN was trained using 148,936 video frames (31 dysplastic patients, 31 NDBE, two normal esophagus), validated on 25,161 images from 11 patient videos and tested on 264 iscan-1 images from 28 dysplastic and 16 NDBE patients which included expert delineations. To localise targeted biopsies/delineations, a second directly supervised CNN was generated based on expert delineations of 94 dysplastic images from 30 patients. This was tested on 86 i-scan one images from 28 dysplastic patients.

Findings: The indirectly supervised CNN achieved a per image sensitivity in the test set of 91%, specificity 79%, area under receiver operator curve of 93% to detect dysplasia. Per-lesion sensitivity was 100%. Mean assessment speed was 48 frames per second (fps). 97% of targeted biopsy predictions matched expert and histological assessment at 56 fps. The artificial intelligence system performed better than six endoscopists.

Interpretation: Our CNNs classify and localise dysplastic Barrett's Esophagus potentially supporting endoscopists during surveillance.

Keywords: AI; Barrett's Esophagus; CAD; CNN; artificial intelligence; computer aided detection; convolutional neural networks; early detection; early neoplasia; neoplasia.

FIGURE 1

Breakdown of the data set in the classification/segmentation models and the potential importance of each model output in the computer aided detection (CAD) system. CAD; Computer aided detection, *In one patient, the video segment of esophagus was split into two segments: dysplastic and NDBE. The former was used for training and the latter for testing

FIGURE 2

AUC performance of the classifier algorithm on iscan‐1 (a) and unenhanced white light (WL) (b)

FIGURE 3

Heat map outputs from the classifier model trained on video frame segments without delineations using an indirectly supervised approach. (a) Original image, (b) expert delineation, (c) heat map generated by the classifier. On the heat maps, the pixels are coloured based on their dysplastic content according to the model. Red areas (closer to 1), show the most likely dysplastic pixels and therefore optimal area for a targeted biopsy

FIGURE 4

Images with BE dysplasia (a) and targeted biopsy and delineation predictions relative to the expert ground truth (b) by the Artificial intelligence (AI) system. Delineations (Green and purple outline) = 2 different expert delineations. Blue shaded delineation = delineation prediction by the convolutional neural network (CNN). Orange and red dot = point of interest/targeted biopsy predicitons by the AI system based on scenario 2 (Table 2)