Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program

Abstract

Introduction: Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption.

Methods: In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters.

Results: From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance.

Conclusion: DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.

Keywords: Artificial intelligence; Cost-utility analysis; Diabetic retinopathy; Health economics; Public health.

Fig. 1

Structure of the cost utility analysis model. A Decision tree representing diabetic retinopathy screening options for patients with diabetes; B Markov model structure representing the clinical progression of diabetic retinopathy (Adapted from the Markov model of Ben et al. [12]). BB bilateral blindness, DME diabetic macular edema, DR diabetic retinopathy, STDR sight-threatening diabetic retinopathy. Non-STDR includes mild and moderate non-proliferative diabetic retinopathy (NPDR) without DME. Severe NPDR and proliferative diabetic retinopathy (PDR) including DME were defined as the STDR health state

Fig. 2

Summary of details of screening and treatment costs. BB bilateral blindness, DL deep learning, DR diabetic retinopathy, DME diabetic macular edema, HG trained human graders, STDR sight-threatening diabetic retinopathy, THB Thai baht. Unit cost of treatment of DME includes cost of bevacizumab and intravitreal administration listed in Table 1. Unit cost of treatment of STDR includes cost of laser photocoagulation and vitrectomy (not shown in this figure but shown in Table 1). Treatment for DME includes cost of outpatient service, bevacizumab, intravitreal injection, and macular imaging by optical coherence tomography. Treatment for STDR in the first year covers the cost of outpatient service and cost of laser photocoagulation, or cost of vitrectomy and inpatient service

Fig. 3

Total and disaggregated costs for the two screening strategies (HG and DL). The costs are for the cost-utility analysis at base case from both societal and provider perspectives. BB bilateral blindness, DL deep learning, DME diabetic macular edema, HG trained human graders, STDR sight-threatening diabetic retinopathy. Costs of treatment of DME and STDR without DME are presented separately. We assumed no direct medical costs for bilateral blindness; all the values are Thai baht in 2020

Fig. 4

Cost-effectiveness plane (CEP). DL deep learning, HG human graders, ICER incremental cost-effectiveness ratio, QALY quality-adjusted life-year, THB Thai baht

Fig. 5

One-way sensitivity analysis. Tornado diagram from one-way sensitivity analysis showing the percentage changes in incremental net monetary benefit (iNMB) of DL screening versus HG screening from the base case attributable to the change of each parameter. ATB antibiotics, BB bilateral blindness, CVD cardiovascular diseases, DL deep learning, DM diabetic mellitus, DME diabetic macular edema, DR diabetic retinopathy, HG human grader, IVB intravitreal bevacizumab, STDR sight-threatening diabetic retinopathy, THB Thai baht. Labels on the chart (next to bars) indicate input values (minimum and maximum) of each parameter; all the values of costs are Thai baht in 2020

Fig. 6

Cost-effectiveness acceptability curves. The curves compare the probabilities of being cost-effective at different willingness to pay of screening using HG and DL in the base case scenario; all the values are Thai baht in 2020. DL deep learning, HG human graders, QALY quality-adjusted life-year, THB Thai baht

Fig. 7

Additional sensitivity analyses. For figure (A), the net monetary benefit (NMB) of each screening modality was calculated from total QALY of the intervention multiplied by the Thai willingness-to-pay threshold (160,000 THB per QALY) and then subtracting the total cost of the intervention. Incremental NMB (iNMB) was the difference between NMB of screening using DL and HG. Values in the table are iNMB of screening using DL and HG. Green cells represent the scenarios when the NMB of DL is greater than the NMB of HG (DL is preferred to HG for the screening). The uptake and compliance rates of HG screening were fixed as base case at 50% and 60%, respectively. Figure (B) shows iNMB between DL and HG screenings increases from zero when the compliance rate of DL screening is more than 44%