Estimating Retinal Sensitivity Using Optical Coherence Tomography With Deep-Learning Algorithms in Macular Telangiectasia Type 2

Abstract

Importance: As currently used, microperimetry is a burdensome clinical testing modality for testing retinal sensitivity requiring long testing times and trained technicians.

Objective: To create a deep-learning network that could directly estimate function from structure de novo to provide an en face high-resolution map of estimated retinal sensitivity.

Design, setting, and participants: A cross-sectional imaging study using data collected between January 1, 2016, and November 30, 2017, from the Natural History Observation and Registry of macular telangiectasia type 2 (MacTel) evaluated 38 participants with confirmed MacTel from 2 centers.

Main outcomes and measures: Mean absolute error of estimated compared with observed retinal sensitivity. Observed retinal sensitivity was obtained with fundus-controlled perimetry (microperimetry). Estimates of retinal sensitivity were made with deep-learning models that learned on superpositions of high-resolution optical coherence tomography (OCT) scans and microperimetry results. Those predictions were used to create high-density en face sensitivity maps of the macula. Training, validation, and test sets were segregated at the patient level.

Results: A total of 2499 microperimetry sensitivities were mapped onto 1708 OCT B-scans from 63 eyes of 38 patients (mean [SD] age, 74.3 [9.7] years; 15 men [39.5%]). The numbers of examples for our algorithm were 67 899 (103 053 after data augmentation) for training, 1695 for validation, and 1212 for testing. Mean absolute error results were 4.51 dB (95% CI, 4.36-4.65 dB) when using linear regression and 3.66 dB (95% CI, 3.53-3.78 dB) when using the LeNet model. Using a 49.9 million-variable deep-learning model, a mean absolute error of 3.36 dB (95% CI, 3.25-3.48 dB) of retinal sensitivity for validation and test was achieved. Correlation showed a high degree of agreement (Pearson correlation r = 0.78). By paired Wilcoxon rank sum test, our model significantly outperformed these 2 baseline models (P < .001).

Conclusions and relevance: High-resolution en face maps of estimated retinal sensitivities were created in eyes with MacTel. The maps were of unequalled resolution compared with microperimetry and were able to correctly delineate functionally healthy and impaired retina. This model may be useful to monitor structural and functional disease progression and has potential as an objective surrogate outcome measure in investigational trials.

Figure 1.. Deep-Learning Model Structure and Learning Curves

A, Schematic of deep-learning model using a total of 49.9 million variables in 7 convolutional (CN) and 2 fully connected (FC) layers with rectified linear unit (ReLU) activation. The first number after CN refers to the number of filters and the second set of numbers refers to the filter sizes. The number after FC refers to the number of neurons. Stride was set to 1 on all layers. B, Learning curves for training and validation with mean squared error as the loss function. Note that we applied data augmentation to training data, but not to validation data. Max indicates maximum.

Figure 2.. Comparison of Observed vs Estimated Retinal Sensitivity

A, Scatterplot showing the observed vs estimated retinal sensitivity. B, Bland-Altman plot showing the difference. The solid gray line represents mean difference obtained across the range of values (mean absolute error), whereas the blue dashed lines are the 95% CIs.

Figure 3.. Microperimetry Results and Predicted High-Resolution Sensitivity Maps

Three patients with MacTel with varying level of disease severity were chosen from the test set (I, II, III). A, The B-scans that crossed the microperimetry stimuli (green line) were chosen. B, The sensitivity scores were overlaid with those B-scans. The color of those overlaid stimuli was manually corrected, resulting in a discrete color scale for each sensitivity score. After deep learning, the model was able to generate a sensitivity estimation line (below optical coherence tomography B-scan), which showed a good correlation with the observed sensitivities. C, Those prediction lines were stitched together to create high-resolution en face sensitivity estimation maps. D, Activation maps of the B-scan, highlighting the regions that the model used for its estimations. Hot areas (red) corresponded to areas most important for the model to estimate normal perimetry values.

Figure 4.. Estimated Progression of Retinal Sensitivity Loss From Longitudinal Optical Coherence Tomographic Scans in 3 Patients With a Follow-up Time of 12 Months

Patient 1 (A), patient 2 (B), and patient 3 (C).The area of estimated functional loss (red) increased over time, in keeping with a slowly progressive disorder. These high-resolution estimated perimetry maps indicate that the model might be useful for disease monitoring.

Figure 5.. Comparison With Baseline Estimation Models

Our model was compared against a linear model as well as the LeNet model. Examples from 3 patients are shown from the test set. A, B, and C represent patients with MacTel with varying degrees of starting disease severity and show the en face estimations for the linear model. D, E, and F, The en face estimations for the LeNet model. G, H, and I, The en face estimations for our model. J, Mean absolute error (95% CIs). Our model achieved significantly better results over these baseline models (P < .001, paired Wilcoxon rank sum test).