Predicting clinical events characterizing the progression of amyotrophic lateral sclerosis via machine learning approaches using routine visits data: a feasibility study

Abstract

Background: Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease that results in death within a short time span (3-5 years). One of the major challenges in treating ALS is its highly heterogeneous disease progression and the lack of effective prognostic tools to forecast it. The main aim of this study was, then, to test the feasibility of predicting relevant clinical outcomes that characterize the progression of ALS with a two-year prediction horizon via artificial intelligence techniques using routine visits data.

Methods: Three classification problems were considered: predicting death (binary problem), predicting death or percutaneous endoscopic gastrostomy (PEG) (multiclass problem), and predicting death or non-invasive ventilation (NIV) (multiclass problem). Two supervised learning models, a logistic regression (LR) and a deep learning multilayer perceptron (MLP), were trained ensuring technical robustness and reproducibility. Moreover, to provide insights into model explainability and result interpretability, model coefficients for LR and Shapley values for both LR and MLP were considered to characterize the relationship between each variable and the outcome.

Results: On the one hand, predicting death was successful as both models yielded F1 scores and accuracy well above 0.7. The model explainability analysis performed for this outcome allowed for the understanding of how different methodological approaches consider the input variables when performing the prediction. On the other hand, predicting death alongside PEG or NIV proved to be much more challenging (F1 scores and accuracy in the 0.4-0.6 interval).

Conclusions: In conclusion, predicting death due to ALS proved to be feasible. However, predicting PEG or NIV in a multiclass fashion proved to be unfeasible with these data, regardless of the complexity of the methodological approach. The observed results suggest a potential ceiling on the amount of information extractable from the database, e.g., due to the intrinsic difficulty of the prediction tasks at hand, or to the absence of crucial predictors that are, however, not currently collected during routine practice.

Keywords: Amyotrophic lateral sclerosis; Explainability; Logistic regression; Multi-layer perceptron.

Fig. 1

LR coefficients for the 10 most impactful variables of the death prediction model. Positive coefficients (right side of the axis) are associated with factors that may increase the likelihood of death. Negative coefficients are associated with factors that may decrease the death probability

Fig. 2

Pearson’s correlation coefficients for all variables combinations. The color blue is associated with a positive correlation meanwhile the color red signals a negative correlation

Fig. 3

SHAP values distributions for the 10 most impactful variables of the LR death model. Positive SHAP values (right side of the figure) are associated with an increase in death probability. Negative SHAP values are associated with a decrease in death probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values

Fig. 4

SHAP values distributions for the 10 most impactful variables of the MLP death model. Positive SHAP values (right side of the figure) are associated with an increase in death probability. Negative SHAP values are associated with a decrease in death probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values

Fig. 5

LR coefficients for the 10 most impactful variables of the death or PEG model. Positive coefficients (right side of the axis) are associated with factors that may increase the likelihood of the outcome. Negative coefficients are associated with factors that may decrease the outcome probability

Fig. 6

SHAP values distributions for the 10 most impactful variables of the LR death or PEG model. Positive SHAP values (right side of the figure) are associated with an increase in outcome probability. Negative SHAP values are associated with a decrease in outcome probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values

Fig. 7

SHAP values distributions for the 10 most impactful variables of the MLP death or PEG model. Positive SHAP values (right side of the figure) are associated with an increase in outcome probability. Negative SHAP values are associated with a decrease in outcome probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values

Fig. 8

LR coefficients for the 10 most impactful variables of the death or NIV model. Positive coefficients (right side of the axis) are associated with factors that may increase the likelihood of the outcome. Negative coefficients are associated with factors that may decrease the outcome probability

Fig. 9

SHAP values distributions for the 10 most impactful variables of the LR death or NIV model. Positive SHAP values (right side of the figure) are associated with an increase in outcome probability. Negative SHAP values are associated with a decrease in outcome probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values

Fig. 10

SHAP values distributions for the 10 most impactful variables of the MLP death or NIV model. Positive SHAP values (right side of the figure) are associated with an increase in outcome probability. Negative SHAP values are associated with a decrease in outcome probability. Distributions are color-coded, red portions are associated with high variable values while blue portions with low variable values