The Value of Automated Diabetic Retinopathy Screening with the EyeArt System: A Study of More Than 100,000 Consecutive Encounters from People with Diabetes

Abstract

Background: Current manual diabetic retinopathy (DR) screening using eye care experts cannot scale to screen the growing population of diabetes patients who are at risk for vision loss. EyeArt system is an automated, cloud-based artificial intelligence (AI) eye screening technology designed to easily detect referral-warranted DR immediately through automated analysis of patient's retinal images. Methods: This retrospective study assessed the diagnostic efficacy of the EyeArt system v2.0 analyzing 850,908 fundus images from 101,710 consecutive patient visits, collected from 404 primary care clinics. Presence or absence of referral-warranted DR (more than mild nonproliferative DR [NPDR]) was automatically detected by the EyeArt system for each patient encounter, and its performance was compared against a clinical reference standard of quality-assured grading by rigorously trained certified ophthalmologists and optometrists. Results: Of the 101,710 visits, 75.7% were nonreferable, 19.3% were referable to an eye care specialist, and in 5.0%, the DR level was unknown as per the clinical reference standard. EyeArt screening had 91.3% (95% confidence interval [CI]: 90.9-91.7) sensitivity and 91.1% (95% CI: 90.9-91.3) specificity. For 5446 encounters with potentially treatable DR (more than moderate NPDR and/or diabetic macular edema), the system provided a positive "refer" output to 5363 encounters achieving sensitivity of 98.5%. Conclusions: This study captures variations in real-world clinical practice and shows that an AI DR screening system can be safe and effective in the real world. This study demonstrates the value of this easy-to-use, automated tool for endocrinologists, diabetologists, and general practitioners to address the growing need for DR screening and monitoring.

Keywords: Artificial intelligence; Automation; Diabetic retinopathy; Screening.

FIG. 1.

Distribution of patients with diabetes in REVERE 100k study. Around 4.9% of the encounters were excluded because of the lack of reference standard due to inadequate image quality. EyeArt system analyzed the remaining encounters, resulting in the following distribution: nonreferable DED (74.3%), referable DED (24.8%), and nonscreenable (0.9%). Images deemed “nonscreenable” by the EyeArt system were considered referable and included in the data analyses. DED, diabetic eye disease; REVERE, REtrospective Validation of Eyeart in the REal world.

FIG. 2.

The number of nonreferable, referable, and nonscreenable encounters in the total population (N = 101,710), and as a function of dilation status in 53.6% of the encounters that were nonmydriatic and 45.8% of encounters that were mydriatic. The 0.6% of encounters whose dilation status was not known is not shown.

FIG. 3.

The sensitivity, specificity, and treatable DR sensitivity were determined for both mydriatic (n = 46,580) and nonmydriatic encounters (n = 54,481), and for the total population (N = 101,710) using the standard formulas shown in Table 2. DR, diabetic retinopathy.

FIG. 4.

Example of images that are of poor image quality due to one or more of the following: insufficient focus, significant under or over exposed areas, presence of image artifacts like lens smudges, flares, or scratches. Inclusion of such images in an encounter may cause the EyeArt system to flag the encounter as nonscreenable. (Color images are available online.)