Hybrid Fusion of High-Resolution and Ultra-Widefield OCTA Acquisitions for the Automatic Diagnosis of Diabetic Retinopathy

Abstract

Optical coherence tomography angiography (OCTA) can deliver enhanced diagnosis for diabetic retinopathy (DR). This study evaluated a deep learning (DL) algorithm for automatic DR severity assessment using high-resolution and ultra-widefield (UWF) OCTA. Diabetic patients were examined with 6×6 mm2 high-resolution OCTA and 15×15 mm2 UWF-OCTA using PLEX®Elite 9000. A novel DL algorithm was trained for automatic DR severity inference using both OCTA acquisitions. The algorithm employed a unique hybrid fusion framework, integrating structural and flow information from both acquisitions. It was trained on data from 875 eyes of 444 patients. Tested on 53 patients (97 eyes), the algorithm achieved a good area under the receiver operating characteristic curve (AUC) for detecting DR (0.8868), moderate non-proliferative DR (0.8276), severe non-proliferative DR (0.8376), and proliferative/treated DR (0.9070). These results significantly outperformed detection with the 6×6 mm2 (AUC = 0.8462, 0.7793, 0.7889, and 0.8104, respectively) or 15×15 mm2 (AUC = 0.8251, 0.7745, 0.7967, and 0.8786, respectively) acquisitions alone. Thus, combining high-resolution and UWF-OCTA acquisitions holds the potential for improved early and late-stage DR detection, offering a foundation for enhancing DR management and a clear path for future works involving expanded datasets and integrating additional imaging modalities.

Keywords: computer-aided diagnosis; deep learning; diabetic retinopathy classification; multimodal information fusion.

Figure 1

Proposed workflow.

Figure 2

Structure and Flow en-face slices (a,b,e,f) and pre-processed B-scan images (flattened retina) (c,d,g,h) from $6\times 6$ mm${}^2$ SS-OCTA and $15\times 15$ mm${}^2$ SS-OCTA. (a,c) Flow of $15\times 15$ mm${}^2$ SS-OCTA. (b,d) Flow of $6\times 6$ mm${}^2$ SS-OCTA. (e,g) Structure of $15\times 15$ mm${}^2$ SS-OCTA. (f,h) Structure of $6\times 6$ mm${}^2$ SS-OCTA. The area of the $6\times 6$ mm${}^2$ SS-OCTA is in the center of the $15\times 15$ mm${}^2$ SS-OCTA image (red bounding box). The green line in the en-face slice shows the source of the B-scan, and the green line in the B-scan image shows the intercept direction of the en-face slice.

Figure 3

Our proposed data processing approach, where N is 10 for $6\times 6$ mm${}^2$ SS-OCTA and 20 for $15\times 15$ mm${}^2$ SS-OCTA. Predictions were based on the same fusion model as for training. Colored discs indicate the DR severity categories.

Figure 4

An illustration of the three types of multimodal fusion networks: (a) input fusion, (b) feature fusion, (c) decision fusion.

Figure 5

An illustration of hierarchical fusion network for $6\times 6$ mm${}^2$ SS-OCTA Structure and Flow.

Figure 6

The results of the different N times Random Crop methods on the validation set for the input fusion of ResNet with the two SS-OCTA acquisitions.