Deep learning with test-time augmentation for radial endobronchial ultrasound image differentiation: a multicentre verification study

Abstract

Purpose: Despite the importance of radial endobronchial ultrasound (rEBUS) in transbronchial biopsy, researchers have yet to apply artificial intelligence to the analysis of rEBUS images.

Materials and methods: This study developed a convolutional neural network (CNN) to differentiate between malignant and benign tumours in rEBUS images. This study retrospectively collected rEBUS images from medical centres in Taiwan, including 769 from National Taiwan University Hospital Hsin-Chu Branch, Hsinchu Hospital for model training (615 images) and internal validation (154 images) as well as 300 from National Taiwan University Hospital (NTUH-TPE) and 92 images were obtained from National Taiwan University Hospital Hsin-Chu Branch, Biomedical Park Hospital (NTUH-BIO) for external validation. Further assessments of the model were performed using image augmentation in the training phase and test-time augmentation (TTA).

Results: Using the internal validation dataset, the results were as follows: area under the curve (AUC) (0.88 (95% CI 0.83 to 0.92)), sensitivity (0.80 (95% CI 0.73 to 0.88)), specificity (0.75 (95% CI 0.66 to 0.83)). Using the NTUH-TPE external validation dataset, the results were as follows: AUC (0.76 (95% CI 0.71 to 0.80)), sensitivity (0.58 (95% CI 0.50 to 0.65)), specificity (0.92 (95% CI 0.88 to 0.97)). Using the NTUH-BIO external validation dataset, the results were as follows: AUC (0.72 (95% CI 0.64 to 0.82)), sensitivity (0.71 (95% CI 0.55 to 0.86)), specificity (0.76 (95% CI 0.64 to 0.87)). After fine-tuning, the AUC values for the external validation cohorts were as follows: NTUH-TPE (0.78) and NTUH-BIO (0.82). Our findings also demonstrated the feasibility of the model in differentiating between lung cancer subtypes, as indicated by the following AUC values: adenocarcinoma (0.70; 95% CI 0.64 to 0.76), squamous cell carcinoma (0.64; 95% CI 0.54 to 0.74) and small cell lung cancer (0.52; 95% CI 0.32 to 0.72).

Conclusions: Our results demonstrate the feasibility of the proposed CNN-based algorithm in differentiating between malignant and benign lesions in rEBUS images.

Keywords: bronchoscopy; lung cancer.

Figure 1

Flow chart illustrating the progression of the study, depicting the sequential steps involved in the research process.

Figure 2

Flow chart illustrating the process of image analysis. NTUH-BIO, National Taiwan University Hospital Hsin-Chu Branch, Biomedical Park Hospital; NTUH-HC, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu Hospital; NTUH-TPE, National Taiwan University Hospital.

Figure 3

Receiver operating characteristic (ROC) curves showcasing the differentiation performance between malignant and benign lesions using different analysis techniques: (A) traditional method, (B) fine-tuning,(C) test-time augmentation, (D) fine-tuning plus test-time augmentation. NTUH-BIO, National Taiwan University Hospital Hsin-Chu Branch, Biomedical Park Hospital; NTUH-HC, National Taiwan University Hospital Hsin-Chu Branch, Hsinchu Hospital; NTUH-TPE, National Taiwan University Hospital.