Assessment of synthetic post-therapeutic OCT images using the generative adversarial network in patients with macular edema secondary to retinal vein occlusion

Abstract

Aims: The aim of this study is to generate post-therapeutic optical coherence tomography (OCT) images based on pre-therapeutic OCT by using generative adversarial networks (GANs). The synthetic images enable us to predict the short-term therapeutic efficacy of intravitreal injection of anti-vascular endothelial growth factor (VEGF) in retinal vein occlusion (RVO) patients.

Methods: The study involved patients with RVO who received intravitreal anti-VEGF injection from 1 November 2018 to 30 November 2019. The OCT images taken before and shortly after treatment, with an interval of 4-8 weeks, were collected and randomly divided into the training set and test set at a ratio of approximately 3:1. The model is constructed based on the pix2pixHD algorithm, and synthetic OCT images are evaluated in terms of the picture quality, authenticity, the central retinal thickness (CRT), the maximal retinal thickness, the area of intraretinal cystoid fluid (IRC), and the area of subretinal fluid (SRF). Three supporting models, namely, the macular detection model, retinal stratification model, and lesion detection model, were constructed. Segmentation of macular location, retinal structure, and typical lesions were added to the input information. After verifying their accuracy, supporting models were used to detect the CRT, the maximal retinal thickness, IRC area, and SRF area of synthetic OCT images. The output predictive values are compared with real data according to the annotation on the real post-therapeutic OCT images.

Results: A total of 1,140 pairs of pre- and post-therapeutic OCT images obtained from 95 RVO eyes were included in the study, and 374 images were annotated. Of the synthetic images, 88% were considered to be qualified. The accuracy of discrimination of real versus synthetic OCT images was 0.56 and 0.44 for two retinal specialists, respectively. The accuracy to predict the treatment efficacy of CRT, the maximal retinal thickness, IRC area, and SRF area was 0.70, 0.70, 0.92, and 0.78, respectively.

Conclusion: Our study proves that the GAN is a reliable tool to predict the therapeutic efficacy of anti-VEGF injections in RVO patients. Evaluations conducted both qualitatively and quantitatively indicated that our model can generate high-quality post-therapeutic OCT images. Consequently, it has great potential in predicting the treatment efficacy and providing guidance to clinical decision-making.

Keywords: anti-vascular endothelial growth factor; generative adversarial networks; optical coherent tomography; retinal vein occlusion; therapeutic efficacy prediction.

FIGURE 1

A conceptual illustration of the pix2pixHD-based solution used in this study for generating post-therapeutic OCT images from pre-therapeutic OCT images.

FIGURE 2

(A–C) illustrated three cases with image quality issues. A1–C1 represent pre-therapeutic OCT images, A2–C2 represent synthetic post-therapeutic OCT images, and A3–C3 represent the real post-therapeutic OCT images. RPE discontinuity is shown in A2 (green line), retinal neuroepithelium discontinuity is shown in B2 (yellow arrow), and the entire retinal discontinuity is shown in C2 (red arrow).

FIGURE 3

(A–C) Three cases of pre-therapeutic, real post-therapeutic, and synthetic post-therapeutic OCT images. A1–C1 represent pre-therapeutic OCT images, A2–C2 represent synthetic post-therapeutic OCT images, and A3–C3 represent real post-therapeutic OCT images.

FIGURE 4

(A–D) Four cases of macular detection. A1–D1 represent the macular position annotated by retinal specialists (red circle). A2–D2 represent the macular position detected by the model (green circle). A3–D3 show the overlap between the annotated and detected positions.

FIGURE 5

(A–B) Two cases of retinal stratification. A1–B1 represent the original OCT images. A2–B2 represent the retinal stratification annotated by retinal specialists. A3–B3 represent retinal stratification detected by the model. A4–B4 show the overlap between the annotated and detected stratification. Black indicates class 0, green indicates class 1, and red indicates class 2.

FIGURE 6

(A–B) Two cases of lesion detection. A1–B1 represent the original OCT images. A2–B2 represent the lesions annotated by retinal specialists. A3–B3 represent the lesion detected by the model. A4–B4 show the overlap between the annotated and detected lesions. Black indicates class 0, green indicates class 1, and red indicates class 2.