Machine Learning Methods Predict Individual Upper-Limb Motor Impairment Following Therapy in Chronic Stroke

Abstract

Background. Accurate prediction of clinical impairment in upper-extremity motor function following therapy in chronic stroke patients is a difficult task for clinicians but is key in prescribing appropriate therapeutic strategies. Machine learning is a highly promising avenue with which to improve prediction accuracy in clinical practice. Objectives. The objective was to evaluate the performance of 5 machine learning methods in predicting postintervention upper-extremity motor impairment in chronic stroke patients using demographic, clinical, neurophysiological, and imaging input variables. Methods. A total of 102 patients (female: 31%, age 61 ± 11 years) were included. The upper-extremity Fugl-Meyer Assessment (UE-FMA) was used to assess motor impairment of the upper limb before and after intervention. Elastic net (EN), support vector machines, artificial neural networks, classification and regression trees, and random forest were used to predict postintervention UE-FMA. The performances of methods were compared using cross-validated R². Results. EN performed significantly better than other methods in predicting postintervention UE-FMA using demographic and baseline clinical data (median $R_{\mathrm{EN}}^2=0.91,R_{\mathrm{RF}}^2=0.88,R_{\mathrm{ANN}}^2=0.83,R_{\mathrm{SVM}}^2=0.79,R_{\mathrm{CART}}^2=0.70;$P < .05). Preintervention UE-FMA and the difference in motor threshold (MT) between the affected and unaffected hemispheres were the strongest predictors. The difference in MT had greater importance than the absence or presence of a motor-evoked potential (MEP) in the affected hemisphere. Conclusion. Machine learning methods may enable clinicians to accurately predict a chronic stroke patient's postintervention UE-FMA. Interhemispheric difference in the MT is an important predictor of chronic stroke patients' response to therapy and, therefore, could be included in prospective studies.

Keywords: Fugl-Meyer Assessment; chronic stroke; machine learning; predictive models; white matter disconnectivity.

Figure 1.. Root of mean squared error (RMSE) results of regression analysis in predicting post-intervention UE-FMA.

Violin plots show the median, 1^st and 3^rd quartile, minimum and maximum value of the RMSE distribution calculated by five machine learning methods. (Elastic-Net (EN) = green, Random Forest (RF) = yellow, Artificial Neural Network (ANN) = purple, Support Vector Machines (SVM) = blue and Classification And Regression Trees (CART) = pink). The panels represent the RMSE over three sets of input variables (A) clinical, (B) clinical + regional disconnectivity and (C) clinical + pair-wise disconnectivity. A significant difference between the performances of two methods was shown with a star above the violin plots.

Figure 2.. The observed versus predicted post-intervention UE-FMA.

Classification and Regression Tree (A) and Elastic-Net (B) methods were trained on clinical input variables, and the observed and predicted post-intervention UE-FMA are presented. Points represent the average prediction while the bars represent the standard deviation over 100 iterations in the outer loop.

Figure 3.. The importance of variables extracted from the methods using the clinical dataset.

Importance of the clinical variables only (demographics, clinical and neurophysiological measures) for all five machine learning methods. (Elastic-Net (EN) = green, Random Forest (RF) = yellow, Artificial Neural Network (ANN) = purple, Support Vector Machines (SVM) = blue and Classification And Regression Trees (CART) = pink). For visualization purposes, the weights of the variables’ importance are rescaled to be relative to pre-intervention UE-FMA.

Figure 4.. AUC results of classification analysis (minimal clinically important difference < 5.5 versus ≥ 5.5).

Violin plots show the median, 1^st and 3^rd quartile, minimum and maximum value of the AUC distribution calculated by five machine learning methods (Elastic-Net (EN) = green, Random Forest (RF) = yellow, Artificial Neural Network (ANN) = purple, Support Vector Machines (SVM) = blue and Classification And Regression Trees (CART) = pink, Logistic Regression (LR)= gray) using (A) clinical, (B) clinical + regional disconnectivity and (C) clinical + pair-wise disconnectivity datasets. A significant difference between the performances of two methods was shown with a star above the violin plots.