Assessment of automated disease detection in diabetic retinopathy screening using two-field photography

Abstract

Aim: To assess the performance of automated disease detection in diabetic retinopathy screening using two field mydriatic photography.

Methods: Images from 8,271 sequential patient screening episodes from a South London diabetic retinopathy screening service were processed by the Medalytix iGrading™ automated grading system. For each screening episode macular-centred and disc-centred images of both eyes were acquired and independently graded according to the English national grading scheme. Where discrepancies were found between the automated result and original manual grade, internal and external arbitration was used to determine the final study grades. Two versions of the software were used: one that detected microaneurysms alone, and one that detected blot haemorrhages and exudates in addition to microaneurysms. Results for each version were calculated once using both fields and once using the macula-centred field alone.

Results: Of the 8,271 episodes, 346 (4.2%) were considered unassessable. Referable disease was detected in 587 episodes (7.1%). The sensitivity of the automated system for detecting unassessable images ranged from 97.4% to 99.1% depending on configuration. The sensitivity of the automated system for referable episodes ranged from 98.3% to 99.3%. All the episodes that included proliferative or pre-proliferative retinopathy were detected by the automated system regardless of configuration (192/192, 95% confidence interval 98.0% to 100%). If implemented as the first step in grading, the automated system would have reduced the manual grading effort by between 2,183 and 3,147 patient episodes (26.4% to 38.1%).

Conclusion: Automated grading can safely reduce the workload of manual grading using two field, mydriatic photography in a routine screening service.

Figure 1. Receiver operator characteristic (ROC) plot for all the graders who took part in the external arbitration.

The three levels of grader (primary, secondary and arbitration) are shown by ‘*’, ‘+’ and ‘o’ respectively. All the graders were compared against a consensus grading calculated from the results from all the arbitration level graders. Notice even amongst the arbitration level graders there is a range of operating points from high specificity and lower sensitivity to high sensitivity and lower specificity.

Figure 2. Workload reduction versus referable sensitivity for the four automated strategies.

The arrows indicate the 95% confidence intervals on the measurements.