Deep learning-based diagnosis from endobronchial ultrasonography images of pulmonary lesions

Abstract

Endobronchial ultrasonography with a guide sheath (EBUS-GS) improves the accuracy of bronchoscopy. The possibility of differentiating benign from malignant lesions based on EBUS findings may be useful in making the correct diagnosis. The convolutional neural network (CNN) model investigated whether benign or malignant (lung cancer) lesions could be predicted based on EBUS findings. This was an observational, single-center cohort study. Using medical records, patients were divided into benign and malignant groups. We acquired EBUS data for 213 participants. A total of 2,421,360 images were extracted from the learning dataset. We trained and externally validated a CNN algorithm to predict benign or malignant lung lesions. Test was performed using 26,674 images. The dataset was interpreted by four bronchoscopists. The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CNN model for distinguishing benign and malignant lesions were 83.4%, 95.3%, 53.6%, 83.8%, and 82.0%, respectively. For the four bronchoscopists, the accuracy rate was 68.4%, sensitivity was 80%, specificity was 39.6%, PPV was 76.8%, and NPV was 44.2%. The developed EBUS-computer-aided diagnosis system is expected to read EBUS findings that are difficult for clinicians to judge with precision and help differentiate between benign lesions and lung cancers.

Figure 1

Convolutional neural network architecture in this study. Feature extraction using CNN consists of two stages, with each stage consisting of multiple blocks and one pooling layer. The first stage consists of 11 blocks and one pooling layer, while the second consists of 16 blocks and one global average pooling layer. One block consists of a convolution layer (Conv), batch normalization layer (BN), and rectified linear unit (ReLU) function. Conv is a dilated convolution. The size of the kernel is 3. Both the dilation size and padding size are 3. The number of channels of Conv in the first and second stages is 135 and 270, respectively. The classification neural network is composed of a fully connected layer (FC) and a softmax layer. The figure was generated by PlotNeuralNet and modified. PlotNeuralNet v1.0.0 (https://github.com/HarisIqbal88/PlotNeuralNet) is released under the MIT License (https://opensource.org/licenses/mit-license.php).

Figure 2

Data preprocessing flow and analysis data breakdown in this study. Training image dataset were 55,376 images of 76 adenocarcinomas, 27,038 images of 41 squamous cell carcinomas, 5136 images of 10 small cell carcinomas, and 33,518 images of 44 benign lesions. Data augmentation was applied to the training image dataset. Test dataset were 11,650 images of 16 adenocarcinomas, 4473 images of 9 squamous cell carcinomas, 2952 images of 5 small cell carcinomas, and 7599 images of 12 benign lesions. The ratio for training and test datasets was 80:20. AD adenocarcinoma, SCC squamous cell carcinomas, SCLC small cell lung cancer.

Figure 3

Accuracy for each case. For each case, when the ratio of images estimated to be correct was 50% or more, it was judged to be correct. When the endobronchial ultrasound visualization was adjacent to case, the graph showed a sprite pattern.

Figure 4

Visualization techniques; Gradient-weighted Class Activation Mapping. Areas suspected of being malignant are shown in red and areas suspected to be benign in blue.