Predicting disease progression in amyotrophic lateral sclerosis

Abstract

Objective: It is essential to develop predictive algorithms for Amyotrophic Lateral Sclerosis (ALS) disease progression to allow for efficient clinical trials and patient care. The best existing predictive models rely on several months of baseline data and have only been validated in clinical trial research datasets. We asked whether a model developed using clinical research patient data could be applied to the broader ALS population typically seen at a tertiary care ALS clinic.

Methods: Based on the PRO-ACT ALS database, we developed random forest (RF), pre-slope, and generalized linear (GLM) models to test whether accurate, unbiased models could be created using only baseline data. Secondly, we tested whether a model could be validated with a clinical patient dataset to demonstrate broader applicability.

Results: We found that a random forest model using only baseline data could accurately predict disease progression for a clinical trial research dataset as well as a population of patients being treated at a tertiary care clinic. The RF Model outperformed a pre-slope model and was similar to a GLM model in terms of root mean square deviation at early time points. At later time points, the RF Model was far superior to either model. Finally, we found that only the RF Model was unbiased and was less subject to overfitting than either of the other two models when applied to a clinic population.

Interpretation: We conclude that the RF Model delivers superior predictions of ALS disease progression.

Figure 1

Model Development Schematic. Schematic showing the relationship between the PRO‐ACT database, the Emory Clinic data, and the three models developed for testing.

Figure 2

Root‐Mean‐Square Deviation over Time. Plots of root‐mean square deviation (RMSD) at 2‐month intervals for the RF model (black), GLM model (green), and pre‐slope model (red). RMSD was calculated for single patient records within each time window. RF, random forest.

Figure 3

Model Performance at 6 Months. Plots of observed ALSFRS‐R score as a function of predicted score for the pre‐slope, GLM and RF models (A, B, C, respectively). Plotting the predictions with observed ALSFRS‐R score on the y‐axis allows a visual indication of predicted accuracy along the spectrum of disease states. In addition to indications of heteroscedasticity along the spectrum of disease states, it is possible to evaluate global bias such as the tendency to underestimate ALSFRS‐R for patients with particularly low scores. RF, random forest.

Figure 4

Bootstrap Mean Prediction Error. Box and whisker plot of prediction error assessed at 6 months via bootstrap resampling. Mean prediction error was assessed via bootstrap resampling to gain insight into model bias. Mean error (solid horizontal line), standard deviation (box border), and 95% confidence interval (whisker) for the pre‐slope, GLM, and RF models. RF, random forest.