Development and validation of an offline deep learning algorithm to detect vitreoretinal abnormalities on ocular ultrasound

Abstract

Purpose: We describe our offline deep learning algorithm (DLA) and validation of its diagnostic ability to identify vitreoretinal abnormalities (VRA) on ocular ultrasound (OUS).

Methods: Enrolled participants underwent OUS. All images were classified as normal or abnormal by two masked vitreoretinal specialists (AS, AM). A data set of 4902 OUS images was collected, and 4740 images of satisfactory quality were used. Of this, 4319 were processed for further training and development of DLA, and 421 images were graded by vitreoretinal specialists (AS and AM) to obtain ground truth. The main outcome measures were sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under receiver operating characteristic (AUROC).

Results: Our algorithm demonstrated high sensitivity and specificity in identifying VRA on OUS ([90.8%; 95% confidence interval (CI): 86.1-94.3%] and [97.1% (95% CI: 93.7-98.9%], respectively). PPV and NPV of the algorithm were also high ([97.0%; 95% CI: 93.7-98.9%] and [90.8%; 95% CI: 86.2-94.3%], respectively). The AUROC was high at 0.939, and the intergrader agreement was nearly perfect with Cohen's kappa of 0.938. The model demonstrated high sensitivity in predicting vitreous hemorrhage (100%), retinal detachment (97.4%), and choroidal detachment (100%).

Conclusion: Our offline DLA software demonstrated reliable performance (high sensitivity, specificity, AUROC, PPV, NPV, and intergrader agreement) for predicting VRA on OUS. This might serve as an important tool for the ophthalmic technicians who are involved in community eye screening at rural settings where trained ophthalmologists are not available.

Keywords: Artificial intelligence; deep learning; ophthalmic technicians; retina; ultrasound; vitreo retinal; vitreous.

Figure 1

Sample selection at Aravind Eye Hospital Pondicherry and Chennai

Figure 2

Ocular ultrasound scan image showing (a) normal structures, (b) multiple vitreous dot echoes, and attached retina (c) detached retina

Figure 3

Area under the receiver operating curve for the deep learning algorithm

Figure 4

Heat maps highlighting regions with abnormalities detected using the DLA. (a) OUS of rhegmatogenous retinal detachment. (b) Heat map of DLA identifying the site of vitreoretinal abnormalities. (c) OUS of tractional retinal detachment. (d) Heatmap of DLA identifying the abnormal tenting of retina. (e) OUS of choroidal detachment. (f) Heatmap of DLA identifying the site of vitreoretinal abnormalities. (g) OUS of vitreous hemorrhage. (h) Heatmap of DLA identifying the site of vitreoretinal abnormalities