Development of a generative deep learning model to improve epiretinal membrane detection in fundus photography

Abstract

Background: The epiretinal membrane (ERM) is a common retinal disorder characterized by abnormal fibrocellular tissue at the vitreomacular interface. Most patients with ERM are asymptomatic at early stages. Therefore, screening for ERM will become increasingly important. Despite the high prevalence of ERM, few deep learning studies have investigated ERM detection in the color fundus photography (CFP) domain. In this study, we built a generative model to enhance ERM detection performance in the CFP.

Methods: This deep learning study retrospectively collected 302 ERM and 1,250 healthy CFP data points from a healthcare center. The generative model using StyleGAN2 was trained using single-center data. EfficientNetB0 with StyleGAN2-based augmentation was validated using independent internal single-center data and external datasets. We randomly assigned healthcare center data to the development (80%) and internal validation (20%) datasets. Data from two publicly accessible sources were used as external validation datasets.

Results: StyleGAN2 facilitated realistic CFP synthesis with the characteristic cellophane reflex features of the ERM. The proposed method with StyleGAN2-based augmentation outperformed the typical transfer learning without a generative adversarial network. The proposed model achieved an area under the receiver operating characteristic (AUC) curve of 0.926 for internal validation. AUCs of 0.951 and 0.914 were obtained for the two external validation datasets. Compared with the deep learning model without augmentation, StyleGAN2-based augmentation improved the detection performance and contributed to the focus on the location of the ERM.

Conclusions: We proposed an ERM detection model by synthesizing realistic CFP images with the pathological features of ERM through generative deep learning. We believe that our deep learning framework will help achieve a more accurate detection of ERM in a limited data setting.

Keywords: Deep learning; Epiretinal membrane; Fundus photography; Generative adversarial net.

Fig. 1

Representative fundus photographs (FPs) of the abnormal semi-translucent film of fibro-cellular tissues of epiretinal membranes (ERM) with reduced visual acuity and healthy retinas. A FP with ERM from the healthcare center data. B FP with ERM from the external validation data. C FP with healthy retina from the healthcare center data. D FP with healthy retina from the external validation data

Fig. 2

Schematic diagram of the development of deep learning model for epiretinal membrane (ERM) detection. The generative adversarial network (GAN) model augments ERM images with proper diversity and high quality to improve diagnostic performance. After augmenting the training data for ERM, we trained deep learning networks via transfer learning to classify ERM and healthy retinas

Fig. 3

Dataset used in developing and validating the epiretinal membrane detection model in fundus photography. The deep learning models were trained and internally validated using randomly partitioned 80 and 20% of data, respectively. Using the training dataset, GAN models were trained to increase the volume of the ERM dataset for data augmentation. We finally built an ERM detection model based on the GAN augmentation techniques. The two external validation datasets, including RFMiD and JSIEC, represented a real scenario of a check-up center with CFP screening

Fig. 4

Epiretinal membrane image generation using generative AI algorithms. A DCGAN. B CycleGAN. C StyleGAN2

Fig. 5

Synthetic fundus photographs according to latent space changes in the StyleGAN2 model

Fig. 6

Validation results of ROC curves for detection of epiretinal membrane. A Healthcare center dataset. B External dataset 1 (RFMiD). B External dataset 2 (JSIEC)

Fig. 7

Attention maps generated by the Grad-CAM technique from the developed EfficientNetB0 to detect epiretinal membrane. A Healthcare center dataset. B External dataset (RFMiD)