Adaptive Semi-Supervised Intent Inferral to Control a Powered Hand Orthosis for Stroke

Abstract

In order to provide therapy in a functional context, controls for wearable robotic orthoses need to be robust and intuitive. We have previously introduced an intuitive, user-driven, EMG-based method to operate a robotic hand orthosis, but the process of training a control that is robust to concept drift (changes in the input signal) places a substantial burden on the user. In this paper, we explore semi-supervised learning as a paradigm for controlling a powered hand orthosis for stroke subjects. To the best of our knowledge, this is the first use of semi-supervised learning for an orthotic application. Specifically, we propose a disagreement-based semi-supervision algorithm for handling intrasession concept drift based on multimodal ipsilateral sensing. We evaluate the performance of our algorithm on data collected from five stroke subjects. Our results show that the proposed algorithm helps the device adapt to intrasession drift using unlabeled data and reduces the training burden placed on the user. We also validate the feasibility of our proposed algorithm with a functional task; in these experiments, two subjects successfully completed multiple instances of a pick-and-handover task.

Fig. 1.

Top: hand orthosis with multimodal sensing suite. Bottom: stroke subject performing an assisted grasp. Due to abnormal synergies, muscle activation signals change significantly compared to collected training data, a type of concept drift that must be accounted for during intent inferral.

Fig. 2.

Example of the classifiers’ output for subject S5 on one of the testing datasets. (a)(b)(c)(d) Comparison of the ground truth user intent and predicted user intent. If the predicted intent is to relax, the intent from the previous time step is used. Data collecting conditions are labeled on top. DSSM-partial, despite being trained only on the third condition {arm off table motor on}, makes correct predictions on the first two conditions and is able to improve its prediction quality as the algorithm runs. (e)(f)(g) Visualization of the confidence produced by DSSM-partial on the third condition. The blue line shows the ground truth user intent as in (a)(b)(c)(d), and other colored lines are confidence values.

Fig. 3.

Example of subject S5 performing pick-and-handover functional task. (a) The subject is instructed to pick up the wooden block and hand it over to an experimenter. We also provide a button for emergency override of classifier control. The orthosis tendon retracts (hand opens) when the button is pressed and the tendon extends (hand closes) when the button is released. (b) An open signal is detected, the tendon retracts and the subject tries to reach the target object. (c) A close signal is detected, the tendon extends and the subject grasps the block. (d) The subject moves the hand carrying the block to the experimenter. (e) An open signal is detected, the tendon retracts and the subject places the block.