An Image Turing Test on Realistic Gastroscopy Images Generated by Using the Progressive Growing of Generative Adversarial Networks

Abstract

Generative adversarial networks (GAN) in medicine are valuable techniques for augmenting unbalanced rare data, anomaly detection, and avoiding patient privacy issues. However, there were limits to generating high-quality endoscopic images with various characteristics, such as peristalsis, viewpoints, light sources, and mucous patterns. This study used the progressive growing of GAN (PGGAN) within the normal distribution dataset to confirm the ability to generate high-quality gastrointestinal images and investigated what barriers PGGAN has to generate endoscopic images. We trained the PGGAN with 107,060 gastroscopy images from 4165 normal patients to generate highly realistic 512² pixel-sized images. For the evaluation, visual Turing tests were conducted on 100 real and 100 synthetic images to distinguish the authenticity of images by 19 endoscopists. The endoscopists were divided into three groups based on their years of clinical experience for subgroup analysis. The overall accuracy, sensitivity, and specificity of the 19 endoscopist groups were 61.3%, 70.3%, and 52.4%, respectively. The mean accuracy of the three endoscopist groups was 62.4 [Group I], 59.8 [Group II], and 59.1% [Group III], which was not considered a significant difference. There were no statistically significant differences in the location of the stomach. However, the real images with the anatomical landmark pylorus had higher detection sensitivity. The images generated by PGGAN showed highly realistic depictions that were difficult to distinguish, regardless of their expertise as endoscopists. However, it was necessary to establish GANs that could better represent the rugal folds and mucous membrane texture.

Keywords: Gastroscopy image; Generative adversarial networks; Image Turing test; Progressive growing of generative adversarial networks; Synthetic image.

Fig. 1

Process of gastric endoscopy image Turing test, A randomly generated 100 gastroscopy images using Progressive growing of GAN (PGGAN), B randomly extracted 100 images from health checkup data, C Turing test on 19 endoscopists to classify 100 normal images and 100 synthetic images, D Analyzing the diagnostic results of the Turing test

Fig. 2

The training process of Progressive growing of GAN (PGGAN). As the training continues, the generator and discriminator increase the resolution of generated images to 512² pixels. There are 8 sample endoscopic images generated using progressive growing at 512² pixels on the right

Fig. 3

Left Result of differentiating performance difference between the three groups, Right Sensitivity and specificity of visual Turing test of synthesized high-resolution gastroscopy. There is no significant difference in discriminating between the three groups in the above two images

Fig. 4

Sample images of synthesized high-resolution gastroscopy. A Cardia and fundus in retroflexion view. B Lesser curvature and anterior wall of a high body in retroflexion view. C High body and mid body in retroflexion view. D Mid body and low body in forward view. E Angle and antrum in retroflexion view. F Antrum and pylorus in forward view. G Pylorus in forward view. H 2nd portion of duodenum in forward view)

Fig. 5

Comparison of the number of images generated for each part of the stomach. (Total 100 generated images) The gastroscopy images were randomly generated without adjusting the distribution

Fig. 6

Synthetic images that the majority of readers decided were fake. A Irregular transverse folds in the lesser curvature. B Absence of normal gastric folds on the greater curvature of the lower body and scattered hyperemic mucosa and asymmetry between anterior wall and posterior wall of the angle. C Abrupt discontinuation and fusion of gastric folds in the greater curvature. D Absence of normal gastric folds on the greater curvature of the low body and asymmetry between anterior wall and posterior wall of the angle. E Abrupt discontinuation of gastric folds in the greater curvature. F Absence of an endoscope in the esophagogastric junction. G Discontinuation of circular folds on the duodenum. H Abrupt discontinuation and fusion of gastric folds in the greater curvature. I Absence of gastric folds and fundus. J Absence of gastric folds and fundus)