Reliability, validity and discriminant ability of the instrumental indices provided by a novel planar robotic device for upper limb rehabilitation

Abstract

Background: In the last few years, there has been an increasing interest in the use of robotic devices to objectively quantify motor performance of patients after brain damage. Although these robot-derived measures can potentially add meaningful information about the patient's dexterity, as well as be used as outcome measurements after the rehabilitation treatment, they need to be validated before being used in clinical practice. The present work aims to evaluate the reliability, the validity and the discriminant ability of the metrics provided by a novel robotic device for upper limb rehabilitation.

Methods: Forty-eight patients with sub-acute stroke and 40 age-matched healthy subjects were involved in this study. Clinical evaluation included: Fugl-Meyer Assessment for the upper limb, Action Research Arm Test, and Barthel Index. Robotic evaluation of the upper limb performance consisted of 14 measures of motor ability quantifying the dexterity in performing planar reaching movements. Patients were evaluated twice, one day apart, to assess the reliability of the robotic metrics, using the Intraclass Correlation Coefficient. Validity was assessed by analyzing the correlation of the robotic metrics with the clinical scales, by means of the Spearman's Correlation Coefficient. Finally, the ability of the robotic metrics to distinguish between patients with stroke and healthy subjects was investigated with t-tests and the Effect Size.

Results: Reliability was found to be excellent for 12 measures and from moderate to good for the remaining 2. Most of the robotic indices were strongly correlated with the clinical scales, while a few showed a moderate correlation and only one was not correlated with the Barthel Index and weakly correlated with the remain two. Finally, all but one the provided metrics were able to discriminate between the two groups, with large effect sizes for most of them.

Conclusion: We found that all the robotic indices except one provided by a novel robotic device for upper limb rehabilitation are reliable, sensitive and strongly correlated both with motor and disability clinical scales. Therefore, this device is suitable as evaluation tool for the upper limb motor performance of patients with sub-acute stroke in clinical practice.

Trial registration: NCT02879279 .

Keywords: Discriminant ability; Reliability; Robot-mediated evaluation; Stroke; Upper limb; Validity.

Fig. 1

Patient engaged in a rehabilitation session with MOTORE

Fig. 2

The evaluation task of MOTORE. In figure is showed the visual feedback showed to the patients on the screen, together with the position of each target. The white ball indicates the position of the end-effector; the yellow circle indicates the target to be reached. The yellow squares, not showed to the patient during the task, indicate the position of the targets: C is the central target, while the numbers from 1 to 8 indicate the external targets with the sequence of the center-out movements. In addition, the distance of each target from the center is reported

Fig. 3

Bland-Altman plots of the robotic indices assessing the whole task

Fig. 4

Bland-Altman plots of the robotic indices assessing the path length travelled by stroke patients towards each target

Fig. 5

Intra-session reliability analysis in stroke patients: robotic indices assessing the whole task. Blue lines represent the statistical analysis of the first session (test), while green lines represent the statistical analysis of the second session (retest). The symbols *, ** and *** represent a statistically significant difference between repetitions, with a p value less than 0.05, 0.01 and 0.001, respectively

Fig. 6

Intra-session reliability analysis in stroke patients: robotic indices assessing the path length travelled towards each target. Blue lines represent the statistical analysis of the first session (test), while green lines represent the statistical analysis of the second session (retest)