Utilizing human intelligence in artificial intelligence for detecting glaucomatous fundus images using human-in-the-loop machine learning

Abstract

Purpose: For diagnosing glaucomatous damage, we have employed a novel convolutional neural network (CNN) from TrueColor confocal fundus images to conquer the black box dilemma in artificial intelligence (AI). This neural network with CNN architecture with human-in-the-loop (HITL) data annotation helps not only in diagnosing glaucoma but also in predicting and locating detailed signs in the glaucomatous fundus, such as splinter hemorrhages, glaucomatous optic atrophy, vertical glaucomatous cupping, peripapillary atrophy, and retinal nerve fiber layer (RNFL) defect.

Methods: The training was done on a well-curated private dataset of 1,400 high-resolution confocal fundus images, out of which 1,120 images (80%) were used exclusively for training and 280 images (20%) were used exclusively for testing. A custom trained You Only Look Once version 5 (YOLOv5)-based object detection methodology was used to identify the underlying conditions precisely. Twenty-six predefined medical conditions were annotated by a team of humans (comprising two glaucoma specialists and two optometrists) by using the Microsoft Visual Object Tagging Tool (VoTT) tool. The 280 testing images were split into three groups (90,100, and 90 images) for three test runs done once every 15 days.

Results: Test results showed consistent increments in the accuracy, from 94.44% to 98.89%, in predicting the glaucoma diagnosis along with the detailed signs of the glaucomatous fundus.

Conclusion: Utilizing human intelligence in AI for detecting glaucomatous fundus images by using HITL machine learning has never been reported in the literature before. This AI model not only has good sensitivity and specificity in accurate glaucoma predictions but is also an explainable AI, thus overcoming the black box dilemma.

Keywords: Artificial Intelligence; Confocal Fundus Images; Glaucomatous Cupping; HITL; Machine Learning.

Figure 1

(a) Sample fundus photograph of an eye with glaucomatous cupping and retinal nerve fiber layer defect utilized for annotating. (b) Customized labeling of the optic cup (green-dotted area). (c) Customized labeling of bayoneting signs (red-dotted area). (d) Customized labeling of superior notching (blue-dotted area). (e) Customized labeling of the optic disc (pink-dotted area). (f) Customized labeling of peripapillary atrophy (gray-dotted area). (g) Customized annotation of the retinal nerve fiber layer (RNFL) defect (green-dotted area). (h) Complete annotation of a fundus image with glaucomatous changes in the optic nerve head and RNFL region

Figure 2

Image showing the methodology workflow of this study

Figure 3

A sample of the batch size of eight image predictions during training, consisting of class probabilities, objectness scores, and bounding boxes

Figure 4

(a) 2D distribution graph showing the annotations which were repeatedly used depicted as spikes. (b and c) 3D distribution graph showing repeated annotations seen as warmer colors

Figure 5

(a) Image showing the quality of training during the beginning of training with respect to GLoU, objectness. classification, precision, and recall. (b) Image showing the quality of improvement at the end of the training with respect to GLoU, objectness, classification, precision, and recall

Figure 6

(a-d) Image depicting the prediction done by the trained AI module on feeding a new fundus image not previously trained by the tool, after the AI tool has been primed and trained. These images were predicted with diagnosis, severity, and all their detailed findings seen in glaucomatous damage