EyeArt artificial intelligence analysis of diabetic retinopathy in retinal screening events

Abstract

Purpose: Early detection and treatment of diabetic retinopathy (DR) are critical for decreasing the risk of vision loss and preventing blindness. Community vision screenings may play an important role, especially in communities at higher risk for diabetes. To address the need for increased DR detection and referrals, we evaluated the use of artificial intelligence (AI) for screening DR.

Methods: Patient images of 124 eyes were obtained using a 45° Canon Non-Mydriatic CR-2 Plus AF retinal camera in the Department of Endocrinology Clinic (Newark, NJ) and in a community screening event (Newark, NJ). Images were initially classified by an onsite grader and uploaded for analysis by EyeArt, a cloud-based AI software developed by Eyenuk (California, USA). The images were also graded by an off-site retina specialist. Using Fleiss kappa analysis, a correlation was investigated between the three grading systems, the AI, onsite grader, and a US board-certified retina specialist, for a diagnosis of DR and referral pattern.

Results: The EyeArt results, onsite grader, and the retina specialist had a 79% overall agreement on the diagnosis of DR: 86 eyes with full agreement, 37 eyes with agreement between two graders, 1 eye with full disagreement. The kappa value for concordance on a diagnosis was 0.69 (95% CI 0.61-0.77), indicating substantial agreement. Referral patterns by EyeArt, the onsite grader, and the ophthalmologist had an 85% overall agreement: 96 eyes with full agreement, 28 eyes with disagreement. The kappa value for concordance on "whether to refer" was 0.70 (95% CI 0.60-0.80), indicating substantial agreement. Using the board-certified retina specialist as the gold standard, EyeArt had an 81% accuracy (101/124 eyes) for diagnosis and 83% accuracy (103/124 eyes) in referrals. For referrals, the sensitivity of EyeArt was 74%, specificity was 87%, positive predictive value was 72%, and negative predictive value was 88%.

Conclusions: This retrospective cross-sectional analysis offers insights into use of AI in diabetic screenings and the significant role it will play in automated detection of DR. The EyeArt readings were beneficial with some limitations in a community screening environment. These limitations included a decreased accuracy in the presence of cataracts and the functional cost of EyeArt uploads in a community setting.

Keywords: Artificial intelligence; Community vision outreach; Diabetic retinopathy.