Comparison of AI-based retinal fluid monitoring in neovascular age-related macular degeneration with manual assessment by different eye care professionals under optimized conditions

Abstract

Purpose: To investigate whether automated intra- and subretinal fluid (IRF/SRF) volume measurements are equivalent to manual evaluations by eye care professionals from different backgrounds on real-world optical coherence tomography (OCT) images in neovascular age-related macular degeneration (nAMD).

Methods: Routine OCT images (Spectralis, Heidelberg Engineering) were obtained during standard-of-care anti-VEGF treatment for nAMD at a tertiary referral centre. IRF/SRF presence and change (increase/decrease/stability) were assessed without time constraints by five retinologists, three ophthalmology residents, three general ophthalmologists, three orthoptists and three certified readers. Fluid volumes were segmented and quantified using a regulatory-approved AI-based tool (Vienna Fluid Monitor, RetInSight, Vienna, Austria). Sensitivity/specificity (Sen/Spe) for grading fluid presence and kappa agreement were calculated for each group. Their performances in distinguishing between IRF/SRF increase and decrease were assessed using AUCs.

Results: About 124 follow-up visit pairs of 59 eyes with active nAMD were included. Across all five groups, fluid volumes >5 nL were identified with values of 0.81-0.95 (Sen)/0.70-0.91 (Spe) for IRF and 0.89-0.98 (Sen)/0.74-0.90 (Spe) for SRF. Interpretations of IRF changes between -17 nL and +3 nL and SRF changes between -9.30 nL and +6.50 nL were associated with Sen > 0.80 and Spe > 0.87 among all groups. Agreements between the algorithm and groups in grading IRF/SRF presence ranged from κ = 0.69-0.82/0.73-0.79. The AUC for correctly classifying fluid change was >0.89 across all groups.

Conclusion: Eye care professionals with different levels of clinical expertise assessed disease activity on standard OCT images with comparable accuracy. Despite optimizing the methodology and time resources, manual performance did not reach the high level of automated fluid monitoring.

Keywords: automated; grading; manual; neovascular age‐related macular degeneration; optical coherence tomography; retinal fluid.

FIGURE 1

Representative examples of spectral‐domain OCT scans from nAMD patients. B‐scans in the left column are shown without fluid segmentations, in the right with segmentations from the deep learning fluid monitor. Intraretinal fluid is shown in yellow, subretinal fluid in green.

FIGURE 2

Distribution of intra‐ (IRF) and subretinal fluid (SRF) volumes (nL) at visits 1 and 2. ‘X' represents the mean of the data. Outliers are shown as dots outside the whiskers.

FIGURE 3

Classification performance for fluid change of the five professional groups. The receiver operating characteristic (ROC) curves show the classification performance of each professional group in discriminating between eyes with intra‐ (IRF) and subretinal fluid (SRF) increase and decrease. Dashed lines indicate chance performance. Each table in the bottom right corner lists the corresponding area under the curve (AUC) values and their confidence intervals.

FIGURE 4

Example of a grading disagreement between the human graders and the fluid monitor. Spectral‐domain OCT B‐scans of a patient at three corresponding positions (three rows) at two consecutive visits (columns 2 and 3). Although there was a subretinal fluid (SRF) increase of approx. 100 nL from visit 1 to visit 2, as quantified by the fluid monitor (green segmentations), most graders indicated an SRF decrease. This discrepancy might be due to the obvious reduction of a larger SRF pocket seen in the first row as well as an underestimation of multiple narrow and elongated SRF pockets at visit 2, as shown in the exemplary images in rows 2 and 3.