Evaluation of Multiple Machine Learning Models for Predicting Number of Anti-VEGF Injections in the Comparison of AMD Treatment Trials (CATT)

Abstract

Purpose: To apply machine learning models for predicting the number of pro re nata (PRN) injections of antivascular endothelial growth factor (anti-VEGF) for neovascular age-related macular degeneration (nAMD) in two years in the Comparison of AMD (age-related macular degeneration) Treatments Trials.

Methods: The data from 493 eligible participants randomized to PRN treatment of ranibizumab or bevacizumab were used for training (n = 393) machine learning models including support-vector machine (SVM), random forest, and extreme gradient boosting (XGBoost) models. Model performances of prediction using clinical and image data from baseline, weeks 4, 8, and 12 were evaluated by the area under the receiver operating characteristic curve (AUC) for predicting few (≤8) or many (≥19) injections, by R2 and mean absolute error (MAE) for predicting the total number of injections in two years. The best model was selected for final validation on a test dataset (n = 100).

Results: Using training data up to week 12, the models achieved AUCs of 0.79-0.82 and 0.79-0.81 for predicting few and many injections, respectively, with R2 of 0.34-0.36 (MAE = 4.45-4.58 injections) for predicting total injections in two years from cross-validation. In final validation on the test dataset, the SVM model had AUCs of 0.77 and 0.82 for predicting few and many injections, respectively, with R2 of 0.44 (MAE = 3.92 injections). Important features included fluid in optical coherence tomography, lesion characteristics, and treatment trajectory in the first three months.

Conclusions: Machine learning models using loading dose phase data have the potential to predict two-year anti-VEGF demand for nAMD and quantify feature importance for these predictions.

Translational relevance: Prediction of anti-VEGF injections using machine learning models from readily available data, after further validation on independent datasets, has the potential to help optimize treatment protocols and outcomes for nAMD patients in an individualized manner.

Figure 1.

ROC curves for the selected SVM model for predicting (A) few (≤8) and (B) many (≥19) PRN injections in 2 years using demographic and ocular characteristics available at baseline (n = 100), week 4 (n = 98), week 8 (n = 95), and week 12 (n = 89) in the test dataset.

Figure 2.

Relative feature importance for the selected SVM model for predicting PRN injections in 2 years using demographic and ocular characteristics available at baseline (n = 393) and week 12 (n = 352) in the training dataset. The top 10 features by relative importance are displayed for the baseline analyses (A, few (≤8) injections; B, many (≥19) injections; C, number of injections) and week 12 analyses (D, few injections; E, many injections; F, number of injections).