Predicting later categories of upper limb activity from earlier clinical assessments following stroke: an exploratory analysis

Abstract

Background: Accelerometers allow for direct measurement of upper limb (UL) activity. Recently, multi-dimensional categories of UL performance have been formed to provide a more complete measure of UL use in daily life. Prediction of motor outcomes after stroke have tremendous clinical utility and a next step is to explore what factors might predict someone's subsequent UL performance category.

Purpose: To explore how different machine learning techniques can be used to understand how clinical measures and participant demographics captured early after stroke are associated with the subsequent UL performance categories.

Methods: This study analyzed data from two time points from a previous cohort (n = 54). Data used was participant characteristics and clinical measures from early after stroke and a previously established category of UL performance at a later post stroke time point. Different machine learning techniques (a single decision tree, bagged trees, and random forests) were used to build predictive models with different input variables. Model performance was quantified with the explanatory power (in-sample accuracy), predictive power (out-of-bag estimate of error), and variable importance.

Results: A total of seven models were built, including one single decision tree, three bagged trees, and three random forests. Measures of UL impairment and capacity were the most important predictors of the subsequent UL performance category, regardless of the machine learning algorithm used. Other non-motor clinical measures emerged as key predictors, while participant demographics predictors (with the exception of age) were generally less important across the models. Models built with the bagging algorithms outperformed the single decision tree for in-sample accuracy (26-30% better classification) but had only modest cross-validation accuracy (48-55% out of bag classification).

Conclusions: UL clinical measures were the most important predictors of the subsequent UL performance category in this exploratory analysis regardless of the machine learning algorithm used. Interestingly, cognitive and affective measures emerged as important predictors when the number of input variables was expanded. These results reinforce that UL performance, in vivo, is not a simple product of body functions nor the capacity for movement, instead being a complex phenomenon dependent on many physiological and psychological factors. Utilizing machine learning, this exploratory analysis is a productive step toward the prediction of UL performance. Trial registration NA.

Keywords: Accelerometry; Outcome assessments; Rehabilitation; Stroke; Supervised machine learning; Upper extremity.

Fig. 1

Coxcomb charts of the five UL performance categories of the 54 participants in this analysis (categories assigned in Barth et.al 2021). The five UL performance variables are divided into equally segmented wedges on the radial chart and the area of each wedge is proportional to the magnitude of the score on that dimension relative to the sample that created the categories. Each chart illustrates the contribution of the five UL performance variables on a standardized scale and are anchored to the minimum and maximum value of each variable in the prior analysis used to establish the categories. The categories are presented in order of increasing overall UL performance and are named: A minimal activity/rare integration; B minimal activity/limited integration; C moderate activity/moderate integration; D moderate activity/full integration; E high activity/full integration

Fig. 2

Single unpruned decision tree

Fig. 3

Variable importance plot for the six models built with bagging algorithms from different input datasets and tuning parameters. Variable importance here is computed using the mean change in accuracy (x-axis), and is expressed relative to the maximum. Higher values indicate greater importance of the specific predictor in the model and values < 0 indicate these predictors decrease the overall accuracy of the model. The shape represents the algorithm used and color represents the size of the input dataset. The small data set that includes UL clinical measures, the medium sized data set includes UL clinical measures + non-motor clinical measures, and the large sized data set includes UL clinical measures + non-motor clinical measures + demographics. The bagged models were built with all predictors available in the data set and random forests were built with the square root of the number of predictors

Fig. 4

Variable importance plot for the six models built with different input datasets and tuning parameters. Variable importance here is computed using the mean change in the Gini index (x-axis), and is expressed relative to the maximum. The shape represents the algorithm used and color represents the size of the input dataset. The small data set that includes UL clinical measures, the medium sized data set includes UL clinical measures + non-motor clinical measures, and the large sized data set includes UL clinical measures + non-motor clinical measures + demographics. The bagged models were built with all predictors available in the data set and random forests were built with the square root of the number of predictors