Improving internal model strength and performance of prosthetic hands using augmented feedback

Abstract

Background: The loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays.

Methods: Our recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance.

Results: Results obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback.

Conclusions: We extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

Keywords: Augmented feedback; Electromyography; Internal model; Motor learning; Muscles; Performance; Prosthetics; Real-time systems; Sensory feedback; Support vector machines.

Fig. 1

Closing the control loop using audio to augment the visual feedback. Dark blue lines represent the classifier-based control signals, red lines represent the regression-based control signals, and purple lines represent the audio feedback

Fig. 2

Subject controlling a prosthetic hand to grasp-and-lift an instrumented virtual egg without breaking it. The prosthetic hand is controlled using the subject’s myoelectric signals sensed by an electrode array placed on their forearm

Fig. 3

Hand starting pose. a Starting pose for the training and familiarization, adaptation, and JND blocks. Subjects had to only activate the thumb and fingers flexion to grasp the object carefully without breaking it. b Starting pose for the performance test: fingers and thumb are extended, and the thumb is abducted. Subjects had to adduct the thumb and then close the hand to grasp the object and transfer it from one side of a barrier to the other

Fig. 4

Psychophysical test results. a Adaptation rate results showing audio-augmented feedback control strategy enabling higher adaptation to self-generated error than the no-augmented feedback control strategy. b Perception threshold test results showing low JND value when using the audio-augmented controller. c Internal model uncertainty (Pparam) results showing significant reduction in the internal model uncertainty when using the audio-augmented feedback control strategy. Horizontal bars indicate statistical significant difference. NF: No-augmented Feedback. AF: Audio-augmented Feedback

Fig. 5

Successful transfer rate of the instrumented virtual egg from one side of a barrier to the other without breaking it. Subjects had 1.74 times higher successful transfers when using the audio-augmented feedback control strategy than when using the no-augmented feedback control strategy. NF: No-augmented Feedback. AF: Audio-augmented Feedback

Fig. 6

Completion time for successful transfers. Subjects using the no-augmented feedback controller had similar completion time to subjects using the audio-augmented controller. NF: No-augmented Feedback. AF: Audio-augmented Feedback

Fig. 7

Progression of grasp-and-lift trials ranging from the beginning of the task (light gray) to the end of the task (dark gray). Representative data from a single subject during adaptation rate test using (a) the no-augmented feedback control strategy (moderate grasp force changes per trial) and (b) the audio-augmented feedback control strategy (high grasp force changes per trial). The red line in both plots shows the preset breaking force

Fig. 8

Submovements computed from the grasp forces of successful trials from the adaptation rate test for a sample of five subjects. NF: No-augmented Feedback. AF: Audio-augmented Feedback