An Artificial Intelligence System for the Detection of Bladder Cancer via Cystoscopy: A Multicenter Diagnostic Study

Abstract

Background: Cystoscopy plays an important role in bladder cancer (BCa) diagnosis and treatment, but its sensitivity needs improvement. Artificial intelligence has shown promise in endoscopy, but few cystoscopic applications have been reported. We report a Cystoscopy Artificial Intelligence Diagnostic System (CAIDS) for BCa diagnosis.

Methods: In total, 69 204 images from 10 729 consecutive patients from 6 hospitals were collected and divided into training, internal validation, and external validation sets. The CAIDS was built using a pyramid scene parsing network and transfer learning. A subset (n = 260) of the validation sets was used for a performance comparison between the CAIDS and urologists for complex lesion detection. The diagnostic accuracy, sensitivity, specificity, and positive and negative predictive values and 95% confidence intervals (CIs) were calculated using the Clopper-Pearson method.

Results: The diagnostic accuracies of the CAIDS were 0.977 (95% CI = 0.974 to 0.979) in the internal validation set and 0.990 (95% CI = 0.979 to 0.996), 0.982 (95% CI = 0.974 to 0.988), 0.978 (95% CI = 0.959 to 0.989), and 0.991 (95% CI = 0.987 to 0.994) in different external validation sets. In the CAIDS vs urologists' comparisons, the CAIDS showed high accuracy and sensitivity (accuracy = 0.939, 95% CI = 0.902 to 0.964; sensitivity = 0.954, 95% CI = 0.902 to 0.983) with a short latency of 12 seconds, much more accurate and quicker than the expert urologists.

Conclusions: The CAIDS achieved accurate BCa detection with a short latency. The CAIDS may provide many clinical benefits, from increasing the diagnostic accuracy for BCa, even for commonly misdiagnosed cases such as flat cancerous tissue (carcinoma in situ), to reducing the operation time for cystoscopy.

Figure 1.

Flowchart of the study design for the development and validation of the Cystoscopy Artificial Intelligence Diagnostic System (CAIDS). AMUFH = The First Hospital Affiliated to Army Medical University; NJFH = The First Affiliated Hospital of Nanjing Medical University; RJH = Renji Hospital Affiliated to Shanghai Jiao Tong University School of Medicine; STCH = Shantou Central Hospital; SYSMH = Sun Yat-sen University Memorial Hospital; SZSH = Shenzhen Second People’s Hospital.

Figure 2.

Framework of the Cystoscopy Artificial Intelligence Diagnostic System. An overview of the proposed framework. Images were captured under white light from different angles and were saved in jpg format. Preprocessing: normalizes images to meet the requirements of the model and augments each image into 4 transformed versions. Feature extraction: extracts feature maps through a ResNet-101 model pretrained on ImageNet. Pyramid pooling module: collects more information by concatenating feature maps at different scales in parallel through a 4-level pyramid. CONV = convolutional layer; CONCAT = concat layer.

Figure 3.

Diagnostic performance of the CAIDS in different validation sets. The ROC curves were generated by plotting sensitivity against specificity. The PR curves were generated by plotting recall (also known as sensitivity) against precision (also known as the positive predictive rate). A) ROC curves and their 95% confidence intervals (CIs) in the validation sets; B) PR curves and their 95% CIs in the validation sets; C) ROC curve and its 95% CIs in the validation set of complex cases; D) PR curve and its 95% CIs in the validation set of complex cases; E) performance of the CAIDS compared with urologists with different levels of experience (expert urologists, competent urologists, and trainees; with and without the guidance of the CAIDS) in identifying BCa in a subgroup analysis of complex lesions. AMUFH = The First Hospital Affiliated to Army Medical University; AUC = area under the ROC curve; AUPRC = area under the PR curve; BCa = bladder cancer; CAIDS = Cystoscopy Artificial Intelligence Diagnostic System; NJFH = The First Affiliated Hospital of Nanjing Medical University; PR = precision recall curve; RJH = Renji Hospital Affiliated to Shanghai Jiao Tong University School of Medicine; ROC = receiver operating characteristic; STCH = Shantou Central Hospital; SYSMH = Sun Yat-sen University Memorial Hospital; SYSMH = Sun Yat-sen University Memorial Hospital; SZSH = Shenzhen Second People’s Hospital.