Predicting Visual Acuity Responses to Anti-VEGF Treatment in the Comparison of Age-related Macular Degeneration Treatments Trials Using Machine Learning

Abstract

Purpose: To evaluate multiple machine learning (ML) models for predicting 2-year visual acuity (VA) responses to anti-vascular endothelial growth factor (anti-VEGF) treatment in the Comparison of Age-related Macular Degeneration (AMD) Treatments Trials (CATT) for patients with neovascular AMD (nAMD).

Design: Secondary analysis of public data from a randomized clinical trial.

Participants: A total of 1029 CATT participants who completed 2 years of follow-up with untreated active nAMD and baseline VA between 20/25 and 20/320 in the study eye.

Methods: Five ML models (support vector machine, random forest, extreme gradient boosting, multilayer perceptron neural network, and lasso) were applied to clinical and image data from baseline and weeks 4, 8, and 12 for predicting 4 VA outcomes (≥ 15-letter VA gain, ≥ 15-letter VA loss, VA change from baseline, and actual VA) at 2 years. The CATT data from 1029 participants were randomly split for training (n = 717), from which the models were trained using 10-fold cross-validation, and for final validation on a test data set (n = 312).

Main outcome measures: Performances of ML models were assessed by R² and mean absolute error (MAE) for predicting VA change from baseline and actual VA at 2 years, by the area under the receiver operating characteristic curve (AUC) for predicting ≥ 15-letter VA gain and loss from baseline.

Results: Using training data up to week 12, the ML models from cross-validation achieved mean R² of 0.24 to 0.29 (MAE = 9.1-9.8 letters) for predicting VA change and 0.37 to 0.41 (MAE = 9.3-10.2 letters) for predicting actual VA at 2 years. The mean AUCs for predicting ≥ 15-letter VA gain and loss at 2 years was 0.84 to 0.85 and 0.58 to 0.73, respectively. In final validation on the test data set up to week 12, the models had an R² of 0.33 to 0.38 (MAE = 8.9-9.9 letters) for predicting VA change, an R² of 0.37 to 0.45 (MAE = 8.8-10.2 letters) for predicting actual VA at 2 years, and AUCs of 0.85 to 0.87 and 0.67 to 0.79 for predicting ≥ 15-letter VA gain and loss, respectively.

Conclusions: Machine learning models have the potential to predict 2-year VA response to anti-VEGF treatment using clinical and imaging features from the loading dose phase, which can aid in decision-making around treatment protocols for patients with nAMD.

Financial disclosure(s): The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Keywords: Age-related macular degeneration; Anti-VEGF therapy; Machine learning; Prediction; Visual acuity.

Figure 5:

ROC curves for predicting ≥15-letter VA gain and ≥15-letter VA loss from baseline at 2 years in the test dataset using demographic and ocular characteristics for the SVM (A, ≥15-letter VA gain; B, ≥15-letter VA loss), random forest (C, ≥15-letter VA gain; D, ≥15-letter VA loss), XGBoost (E, ≥15-letter VA gain; F, ≥15-letter VA loss), and MLP (G, ≥15-letter VA gain; H, ≥15-letter VA loss) models.

Figure 6:

Relative feature importance for predicting VA change from baseline and actual VA at 2 years in the training dataset using demographic and ocular characteristics available at week 12 for the SVM (A, VA change; B, actual VA), random forest (C, VA change; D, actual VA), XGBoost (E, VA change; F, actual VA), MLP (G, VA change; H, actual VA), and lasso (I, VA change; J, actual VA) models. The top 10 features by relative importance are displayed.

Figure 7:

Relative feature importance for predicting ≥15-letter VA gain and ≥15-letter VA loss from baseline at 2 years in the training dataset using demographic and ocular characteristics available at week 12 for the SVM (A, ≥15-letter VA gain; B, ≥15-letter VA loss), random forest (C, ≥15-letter VA gain; D, ≥15-letter VA loss), XGBoost (E, ≥15-letter VA gain; F, ≥15-letter VA loss), and MLP (G, ≥15-letter VA gain; H, ≥15-letter VA loss) models. The top 10 features by relative importance are displayed.