Multi-task deep learning-based survival analysis on the prognosis of late AMD using the longitudinal data in AREDS

Abstract

Age-related macular degeneration (AMD) is the leading cause of vision loss. Some patients experience vision loss over a delayed timeframe, others at a rapid pace. Physicians analyze time-of-visit fundus photographs to predict patient risk of developing late-AMD, the most severe form of AMD. Our study hypothesizes that 1) incorporating historical data improves predictive strength of developing late-AMD and 2) state-of-the-art deep-learning techniques extract more predictive image features than clinicians do. We incorporate longitudinal data from the Age-Related Eye Disease Studies and deep-learning extracted image features in survival settings to predict development of late- AMD. To extract image features, we used multi-task learning frameworks to train convolutional neural networks. Our findings show 1) incorporating longitudinal data improves prediction of late-AMD for clinical standard features, but only the current visit is informative when using complex features and 2) "deep-features" are more informative than clinician derived features. We make codes publicly available at https://github.com/bionlplab/AMD_prognosis_amia2021.

Figure 1:

Model Architecture.

Figure 2:

t-SNE plots of the fine-tuned deep features generated from the visits used in the survival analysis (years 0, 2, and 3). Colors indicate time to reach late-AMD. Dark red dots are soon to reach late-AMD. White dots will not reach for many years. Gray spots show the overall distribution of image features censored and uncensored patients.

Figure 3:

t-SNE plots of the feature vectors generated from the testing set used to evaluate the multi-task learning classifier models. Coloring indicates either the true (Reading Center Gradings) or predicted (Model Prediction) values for classifying drusen size and pigmentation abnormalities.