Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection

Abstract

Background/aims: The aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus images for improved retinopathy detection.

Methods: For eligible cataract patients, preoperative and postoperative colour fundus photographs (CFP) and ultra-wide field (UWF) images were captured. Then, both the original CycleGAN and a modified CycleGAN (C²ycleGAN) framework were adopted for image generation and quantitatively compared using Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Additionally, CFP and UWF images from another cataract cohort were used to test model performances. Different panels of ophthalmologists evaluated the quality, authenticity and diagnostic efficacy of the generated images.

Results: A total of 959 CFP and 1009 UWF image pairs were included in model development. FID and KID indicated that images generated by C²ycleGAN presented significantly improved quality. Based on ophthalmologists' average ratings, the percentages of inadequate-quality images decreased from 32% to 18.8% for CFP, and from 18.7% to 14.7% for UWF. Only 24.8% and 13.8% of generated CFP and UWF images could be recognised as synthetic. The accuracy of retinopathy detection significantly increased from 78% to 91% for CFP and from 91% to 93% for UWF. For retinopathy subtype diagnosis, the accuracies also increased from 87%-94% to 91%-100% for CFP and from 87%-95% to 93%-97% for UWF.

Conclusion: Digital ray could generate realistic postoperative CFP and UWF images with enhanced quality and accuracy for overall detection and subtype diagnosis of retinopathies, especially for CFP.\ TRIAL REGISTRATION NUMBER: This study was registered with ClinicalTrials.gov (NCT05491798).

Keywords: Imaging; Lens and zonules; Retina.

Figure 1. Overall study design. (A) Preoperative and postoperative fundus images, including CFP and UWF, were collected from enrolled cataract patients to train digital ray using CycleGAN. (B) CFP and UWF images collected from another cohort of cataract patients were used to test the trained model. (C) Both preoperative and generated fundus images were labelled by three groups of ophthalmologists for clinical evaluation. Annotation results regarding quality, authenticity and diagnostic efficacy of the images were analysed. AMD, age-related macular degeneration; CFP, colour fundus photography; DR, diabetic retinopathy; PM, pathological myopia; RD, retinal detachment; UWF, ultra-wide field.

Figure 2. Clinical evaluation results of digital ray by ophthalmologists of different levels of expertise. Quality evaluation by different groups of ophthalmologists using preoperative and generated CFP (A) and UWF (B) images. The discrimination rate between postoperative and generated images in CFP (C) and UWF (D) group. Overall diagnostic accuracy by different groups of ophthalmologists using preoperative and generated CFP (E) and UWF (F) images. *p<0.05, **p<0.001 for comparison between preoperative images and generated images using McNemar’s test. CFP, colour fundus photography; UWF, ultra-wide field.

Figure 3. Receiver operation characteristic curves for retinopathy detection and subtype diagnosis using CFP and UWF images. This figure summarises ROCs for retinopathy detection (A, B) as well as subtype diagnosis (C, D) using preoperative and generated CFP and UWF images based on all doctors’ average ratings. AMD, age-related macular degeneration; AUC, area under the curve; CFP, colour fundus photography; DR, diabetic retinopathy; PM, pathological myopia; RD, retinal detachment; ROC, receiver operation characteristic curves; UWF, ultra-wide field.