Effectiveness of data-augmentation on deep learning in evaluating rapid on-site cytopathology at endoscopic ultrasound-guided fine needle aspiration

Abstract

Rapid on-site cytopathology evaluation (ROSE) has been considered an effective method to increase the diagnostic ability of endoscopic ultrasound-guided fine needle aspiration (EUS-FNA); however, ROSE is unavailable in most institutes worldwide due to the shortage of cytopathologists. To overcome this situation, we created an artificial intelligence (AI)-based system (the ROSE-AI system), which was trained with the augmented data to evaluate the slide images acquired by EUS-FNA. This study aimed to clarify the effects of such data-augmentation on establishing an effective ROSE-AI system by comparing the efficacy of various data-augmentation techniques. The ROSE-AI system was trained with increased data obtained by the various data-augmentation techniques, including geometric transformation, color space transformation, and kernel filtering. By performing five-fold cross-validation, we compared the efficacy of each data-augmentation technique on the increasing diagnostic abilities of the ROSE-AI system. We collected 4059 divided EUS-FNA slide images from 36 patients with pancreatic cancer and nine patients with non-pancreatic cancer. The diagnostic ability of the ROSE-AI system without data augmentation had a sensitivity, specificity, and accuracy of 87.5%, 79.7%, and 83.7%, respectively. While, some data-augmentation techniques decreased diagnostic ability, the ROSE-AI system trained only with the augmented data using the geometric transformation technique had the highest diagnostic accuracy (88.2%). We successfully developed a prototype ROSE-AI system with high diagnostic ability. Each data-augmentation technique may have various compatibilities with AI-mediated diagnostics, and the geometric transformation was the most effective for the ROSE-AI system.

Figure 1

Procedures to establish the ROSE-AI system in this study. At first, we collected EUS-FNA slides with Diff-Quik stained. Next, using a virtual slide scan, EUS-FNA slides were digitized as whole-slide images (WSI). The WSIs were then divided into smaller images (256 × 256 pixels), and the divided images were classified and annotated as those with and without cancer cells. Data augmentation was performed to increase the number of original data points. The ROSE-AI system was established using a transformer-based model, and we performed five-fold cross-validation.

Figure 2

Examples of data augmentation techniques. We used three data augmentation techniques: geometric transformations, color-space transformations, and kernel filtering. Geometric transformations included perspective transformations, rotation, flipping, Gaussian noise, and cropping. Color-space transformations included brightness, contrast, and saturation adjustments. Gaussian blur was applied as Kernel filtering.

Figure 3

Representative images of the ROSE-AI system detecting cancer cells. A layer-wise relevance propagation (LRP) method was used to visualize and evaluate the ROSE-AI system. (a) Original image. (b) Enhanced image by LRP method: ROSE-AI system focuses on the red area.

Figure 4

The ROC curves for diagnosing pancreatic cancer by the ROSE-AI system using different data-augmentation techniques. The diagnostic performance for each data augmentation method is shown. The ROSE-AI system trained only with the augmented data using the geometric transformation technique had the highest diagnostic accuracy (AUC = 0.954). ROC, receiver operating characteristic; AUC, area under the curve; G, geometric transformations; C, color space transformations; K, kernel filtering.