Deep learning in glaucoma with optical coherence tomography: a review

Abstract

Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has made significant breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, applying DL for glaucoma assessment with optical coherence tomography (OCT), including OCT traditional reports, two-dimensional (2D) B-scans, and three-dimensional (3D) volumetric scans, has increasingly raised research interests. Studies have demonstrated that using DL for interpreting OCT is efficient, accurate, and with good performance for discriminating glaucomatous eyes from normal eyes, suggesting that incorporation of DL technology in OCT for glaucoma assessment could potentially address some gaps in the current practice and clinical workflow. However, further research is crucial in tackling some existing challenges, such as annotation standardization (i.e., setting a standard for ground truth labelling among different studies), development of DL-powered IT infrastructure for real-world implementation, prospective validation in unseen datasets for further evaluation of generalizability, cost-effectiveness analysis after integration of DL, the AI "black box" explanation problem. This review summarizes recent studies on the application of DL on OCT for glaucoma assessment, identifies the potential clinical impact arising from the development and deployment of the DL models, and discusses future research directions.

Fig. 1. Posterior segment optical coherence tomography (OCT) not only enables the top view of the retina and optic nerve head, but also captures deeper and three-dimensional (3D) view of the morphological features.

a an example of OCT volumetric optic disc scan as well as corresponding en face fundus image generated by line-scanning ophthalmoscopy; b an example of OCT volumetric macula scan as well as corresponding en face fundus image.

Fig. 2. Illustration of basic process of a deep learning system development and validation.

Usually, a training set is for the network to learn all the features automatically, while a tuning set is a small evaluation set to supervise the real-time performance. A non-overlapping primary validation set (or testing set) is used to test the final performance after training and tuning are done. These three types of sets are usually split from the same one large dataset based on a specific ratio. To further validate model performance on unseen datasets and verify its generalizability, other independent or unseen datasets are needed as external validation sets. A more generally good performance in all validation datasets, including primary and external validations, means higher generalizability of the DL model.

Fig. 3. There were four categories of deep learning (DL) models with different input.

These input were: (a) OCT measurement images extracted from the traditional OCT report, including retinal nerve fibre layer (RNFL) thickness map, RNFL deviation map, optic disc en face fundus image, ganglion cell with inner plexiform layer (GCIPL) thickness map, GCIPL deviation map, and macula en face fundus image; (b) OCT segmentation-free 2D B-scans; (c) OCT segmentation-free 3D volumetric scans; (d) “Machine-to-Machine” approach to predict OCT quantitative measurements, such as RNFL thickness, GCIPL thickness, and Bruch’s Membrane Opening-based minimum rim width (BMO-MRW), from fundus photographs.

Fig. 4. A potential clinical workflow with deployment of deep learning-based clinical support system for glaucoma detection with OCT images in primary, secondary and tertiary settings.

Subjects undergo SD-OCT scanning first to screen for glaucoma, and the images will be the input of the AI system. The technicians will then make referral suggestions based on the output (i.e., refer to ophthalmologists due to “Yes GON”, or observation only due to “No GON”).

Fig. 5. Examples of heatmap generated by class activation map (CAM) for glaucomatous optic neuropathy (GON) detection generated with a previously published DL algorithm [53].

a The cross-sectional view of original OCT optic disc scans, b the en face view of original OCT optic disc scans, and c the corresponding en face fundus image. The feature maps, i.e., the intermediate outputs of the network layers, before the global average pooling layer as well as the parameters of the fully connected layer were taken to obtain the heatmap. The sum of the feature maps weighted by the parameters were taken to generate the CAM. For this particular deep learning model, the red-orange-coloured regions (i.e., retinal nerve fibre layer and neuroretinal rim) have the most discriminative power to differentiate GON.