Ability of artificial intelligence to detect T1 esophageal squamous cell carcinoma from endoscopic videos and the effects of real-time assistance

Abstract

Diagnosis using artificial intelligence (AI) with deep learning could be useful in endoscopic examinations. We investigated the ability of AI to detect superficial esophageal squamous cell carcinoma (ESCC) from esophagogastroduodenoscopy (EGD) videos. We retrospectively collected 8428 EGD images of esophageal cancer to develop a convolutional neural network through deep learning. We evaluated the detection accuracy of the AI diagnosing system compared with that of 18 endoscopists. We used 144 EGD videos for the two validation sets. First, we used 64 EGD observation videos of ESCCs using both white light imaging (WLI) and narrow-band imaging (NBI). We then evaluated the system using 80 EGD videos from 40 patients (20 with superficial ESCC and 20 with non-ESCC). In the first set, the AI system correctly diagnosed 100% ESCCs. In the second set, it correctly detected 85% (17/20) ESCCs. Of these, 75% (15/20) and 55% (11/22) were detected by WLI and NBI, respectively, and the positive predictive value was 36.7%. The endoscopists correctly detected 45% (25-70%) ESCCs. With AI real-time assistance, the sensitivities of the endoscopists were significantly improved without AI assistance (p < 0.05). AI can detect superficial ESCCs from EGD videos with high sensitivity and the sensitivity of the endoscopist was improved with AI real-time support.

Figure 1

System of AI diagnosis in endoscopic videos and representative images of AI detection of ESCC. (a) When the AI detected a cancerous lesion, the AI reviewed the video for 0.5 s (15 frames). If the reviewed section of video included a cancer image in more than 3 frames and the maximum interval from the latest cancer image was 0.1 s (3 frames), the AI diagnosed the lesion as cancer, giving a discovery signal. (b,c) When the AI recognized a cancerous lesion, a frame was displayed in the endoscopic image surrounding the lesion of interest. The AI inserted the image of the recognized cancer on the left side of the monitor, indicating that it diagnosed the lesion as cancerous. (d) When the AI-diagnosed cancer matched the iodine unstained area which was pathologically diagnosed as an ESCC, the AI was considered correct. AI: artificial intelligence, ESCC: esophageal squamous cell carcinoma.

Figure 2

Sensitivity of the AI diagnosis for each case. The sensitivity of the AI diagnosis for each case was slightly higher in WLI than in NBI, but not significantly. * WLI + NBI: when a cancer was diagnosed with either WLI or NBI, we considered that the AI had detected the cancer. AI: artificial intelligence, WLI: white light imaging, NBI: narrow-band imaging.

Figure 3

Examples of false-positive images. The yellow squares indicate areas that were misdiagnosed as cancer. (a) shadow of lumen, (b) EGJ, (c) post ER scar, (d) inflammation. ER: endoscopic resection, EGJ: esophagogastric junction.

Figure 4

Examples of false-negative images. The cancers in the shown images were missed for the following estimated causes. (a) Inflammation of background mucosa, (b) anterior wall lesion, (c) obscure ESCC by WLI, (d) ESCC less than 5 mm. ESCC: esophageal squamous cell carcinoma, WLI: white light imaging.

Figure 5

Sensitivity of the endoscopists for each case. The median sensitivity is represented by the center line in the box, which indicates the IQR. The range is indicated by whiskers. When diagnosed with either WLI or NBI, we considered the endoscopists to have detected the cancers (WLI + NBI). IQR: interquartile range, WLI: white light imaging, NBI: narrow-band imaging. *P < 0.05.