Artificial Intelligence Versus Rules-Based Approach for Segmenting NonPerfusion Area in a DRCR Retina Network Optical Coherence Tomography Angiography Dataset

Abstract

Purpose: Loss of retinal perfusion is associated with both onset and worsening of diabetic retinopathy (DR). Optical coherence tomography angiography is a noninvasive method for measuring the nonperfusion area (NPA) and has promise as a scalable screening tool. This study compares two optical coherence tomography angiography algorithms for quantifying NPA.

Methods: Adults with (N = 101) and without (N = 274) DR were recruited from 20 U.S. sites. We collected 3 × 3-mm macular scans using an Optovue RTVue-XR. Rules-based (RB) and deep-learning-based artificial intelligence (AI) algorithms were used to segment the NPA into four anatomical slabs. For comparison, a subset of scans (n = 50) NPA was graded manually.

Results: The AI method outperformed the RB method in intersection over union, recall, and F1 score, but the RB method has better precision relative to manual grading in all anatomical slabs (all P ≤ 0.001). The AI method had a stronger rank correlation with Early Treatment of Diabetic Retinopathy Study DR severity than the RB method in all slabs (all P < 0.001). NPAs graded using the AI method had a greater area under the receiver operating characteristic curve for diagnosing referable DR than the RB method in the superficial vascular complex, intermediate capillary plexus, and combined inner retina (all P ≤ 0.001), but not in the deep capillary plexus (P = 0.92).

Conclusions: Our results indicate that output from the AI-based method agrees better with manual grading and can better distinguish between clinically relevant DR severity levels than a RB approach using most plexuses.

Figure 1.

DR severity in eyes at first visit. DR severity in eyes at the first visit (570 eyes from 375 participants). Dashed lines indicate cutoff values for referable (ETDRS level ≥ 35) and vision-threatening DR (ETDRS level ≥53). Most scans in this study were of eyes with ETDRS scores between 35 and 47, which belong to referable but non–vision-threatening DR. An ETDRS score of 10 corresponds to data from volunteers from OSHU who do not have diabetes (ETDRS level = 10, no retinopathy). Numbers in or above the bars indicate the number of eyes at each score.

Figure 2.

Manual segmentation and algorithm outputs. Relative to manual segmentation, the RB algorithm fails to identify some regions of the NPA, possibly owing to the relatively high background noise (see especially the DCP and inner retina images). The AI result, on the other hand, broadly agrees with the general location of NPA regions; however, the specific shape of these regions is not necessarily consistent. The AI result also identifies small NPA regions that were not manually segmented truth image (again, compare especially the inner retina images).

Figure 3.

Correlation between AI and RB NPA measurements. The correlation coefficient measures the linear fit of the data. Sample size was 570 eyes (1 scan per eye) of 375 participants. All correlations were significant at an α level of 0.05. The dashed line represents the line of identity. The inner retina spans each of the other layers considered in this study.

Figure 4.

Difference in NPA between AI and RB methods by anatomical slab. Waterfall plots (showing binned averages of 25 measurements for clarity) showing the mean difference (dotted line, with CI given by the shaded region) between AI and RB NPA measurements (calculated as NPA with RB minus NPA from AI with negative values indicating greater NPA with the AI method). Sample size was 819 scans from 570 eyes of 375 participants for all anatomical slabs. All comparisons were significant at an α level of 0.05.

Figure 5.

Correlation between NPA and image quality by method. Correlation between NPA measurements and image quality (SSI) among eyes from participants without diabetes (125 eyes from 101 participants). Ideally, the SSI and NPA would be uncorrelated in this context.

Figure 6.

NPA by DR severity. Box plots of the NPA by DR severity stratified by method (rows) and anatomical slab (slab). Both methods show an overlap between the NPA and disease severity. The inner retina includes each of the slabs considered in this study. The sample size was 127 scans of 127 eyes from 103 participants for nonreferable DR (ETDRS level <35), 490 scans of 361 eyes from 239 participants for referable but non–vision-threatening DR (ETDRS levels 35 to <53), and 202 scans of 171 eyes from 126 participants for vision-threatening DR (ETDRS level ≥53).

Figure 7.

Received operator characteristic curves for referable and vision-threatening DR. Receiver operating characteristic curves for referable and vision-threatening DR diagnosis. The sample size is 570 eyes from 375 participants. The AI method (solid lines) is in general better than the RB approach (dotted lines) at diagnosing these DR severities, in particular when the inner retina images are used for staging.