Effectiveness of artificial intelligence screening in preventing vision loss from diabetes: a policy model

Abstract

The effectiveness of using artificial intelligence (AI) systems to perform diabetic retinal exams ('screening') on preventing vision loss is not known. We designed the Care Process for Preventing Vision Loss from Diabetes (CAREVL), as a Markov model to compare the effectiveness of point-of-care autonomous AI-based screening with in-office clinical exam by an eye care provider (ECP), on preventing vision loss among patients with diabetes. The estimated incidence of vision loss at 5 years was 1535 per 100,000 in the AI-screened group compared to 1625 per 100,000 in the ECP group, leading to a modelled risk difference of 90 per 100,000. The base-case CAREVL model estimated that an autonomous AI-based screening strategy would result in 27,000 fewer Americans with vision loss at 5 years compared with ECP. Vision loss at 5 years remained lower in the AI-screened group compared to the ECP group, in a wide range of parameters including optimistic estimates biased toward ECP. Real-world modifiable factors associated with processes of care could further increase its effectiveness. Of these factors, increased adherence with treatment was estimated to have the greatest impact.

Fig. 1. Expected vision loss per 100,000 vs probability of adhering with treatment for each screening strategy.

a, b Show that as the adherence with recommended metabolic and ophthalmic treatments increases the number of patients with vision loss per 100,000 decreases for both the eye care provider (ECP) and artificial intelligence (AI) screening strategies. However, the decrease in number with vision loss is more marked for the AI vs ECP screening strategy.

Fig. 2. Additional vision loss prevented beyond AI base-case when maximizing processes of care.

Figure 2 shows the additional impact on vision loss prevented beyond the base-case scenario when each of the processes of care are maximized. The largest impact on vision loss is estimated to be from maximizing adherence with ophthalmic treatment, followed by adherence with metabolic treatments. Maximizing effectiveness of current metabolic and ophthalmic treatments has a lower impact.

Fig. 3. Markov model showing the states and transitions relevant to diabetic retinal disease used in the current analysis.

❶Patients with diabetes mellitus presenting to the primary care or endocrine clinic with each of the following states: No Diabetic Retinal Disease, Metabolic Diabetic Retinal Disease, or Ophthalmic Diabetic Retinal Disease. ❷Natural history transitions of diabetic retinal disease. ❸Transitions from untreated to treated diabetic retinal disease. ❹Transitions of treated diabetic retinal disease. The transitions take into account process-of-care metrics i.e., probability of accepting screening and referral in case of a positive screen, probability of disease progression, probability of adhering with recommended treatments. The structure of the Markov model is the same for both screening strategies. Table 1 shows the base-case probabilities and limits of sensitivity analysis for each parameter that are specific to the AI and ECP screening strategies. The details of the transitions specific to each strategy are represented in the decision tree in the supplement (Fig. 3 is preserved and shared on Figshare (https://figshare.com/s/ad7809b8f7010fdf83c9).