A cross-modal feedback scheme for control of prosthetic grasp strength

Abstract

Introduction: Given the lack of haptic feedback inherent in prosthetic devices, a natural and adaptable feedback scheme must be implemented. While multimodal feedback has proven successful in aiding dexterous performance, it can be mentally tasking on the individual. Conversely, cross-modal schemes relying on sensory substitution have proven to be equally effective in aiding task performance without cognitively burdening the user to the same degree.

Objectives: This experiment investigated the effectiveness of the cross-modal feedback scheme through using audio feedback to represent prosthetic grasping strength during dynamic control of a prosthetic hand.

Methods: A total of five individuals participated in two sets of experiments (four subjects in the first, one subject in the second). Participants were asked to control the grasping strength exerted by a prosthetic hand while using real-time audio feedback in order to reach up to three different levels of force within a trial set.

Results: The cross-modal feedback scheme successfully provided users with the robust ability to modulate grasping strength in real-time using only audio feedback.

Conclusion: Audio feedback effectively conveys haptic information to the user of a prosthetic hand. Retention of the training knowledge is evident and can be generalized to perform new (i.e. untrained) tasks.

Keywords: Amputees; limb prosthetics; prosthetic control; sensation simulation/restoration; upper limb prosthetics.

Figure 1.

(a) This schematic represents the different components of the experiment. The subject first sends a command to the PC by pressing a keystroke on the keyboard. The PC interprets the command as either opening or closing the i-Limb, and sends the appropriate command to the i-Limb. The force sensors on the i-Limb capture the applied force (measured in mV), and send both the force reading to the PC and the audio-feedback signal to the subject. The user has visual feedback of the force reading through a graphical user interface (GUI) displayed on the PC monitor. (b) The experimental setup showing the user (top right) controlling the grasping force of the i-Limb robot hand (top left) using the force feedback GUI (bottom left).

Figure 2.

Graphic user interface for the subject. The user is prompted with the force threshold (two horizontal reference lines) and in real time receives feedback of the forces exerted by the prosthetic hand (red solid line). Instances showing the definition of the rise and settling time are shown with dashed vertical lines. The rise time is determined as the time required for the subjects to apply force equivalent to 90% of the final recorded applied force (percent of trail completed=100). The settling time is determined by the time required for the subject to reach and stay within a 5% threshold of the final recorded applied force.

Figure 3.

This figure represents the rise and settling times of Experiment 1 for Subject 1, a representative sample in the experiment. The rise and settling times are broken down into the three combinations. L, M, H, NL, and NM represent “low”, “medium”, “high”, “new-low”, and “new-medium” respectively. Rise time analysis demonstrates an improvement in task performance over time, indicating that with audio feedback alone individuals can effectively perform the grasping task. (a) This figure shows the rise times for Combination 1. The average rise times for the low and new-low force levels lie within ± 5% of 12%. The medium and new-medium average rise times lie approximately within ± 2% of 14%. The average rise for the high forces lie within approximately ± 8% of 29%. (b) This figure shows the settling times for Combination 1. The average settling time for the low and new-low force level lies within ± 20% of 45%. The average medium and new medium lies within ± 5% of 25%. The average high lies within ± 15 of 55%. (c) This figure shows the rise time for Combination 2. The average for the low and new low levels lies within ± 4% of 15%. The average for the medium and new medium lies within ± 3% of 12%. The average for the high force level lies within ± 5% of 25%. (d) This figure shows the settling times for Combination 2. The average settling time for low and new-low levels of force lies within ± 40% of 60%. The average medium and new medium lies within ± 15 of 45%. The average high lies within ± 15% of 50%. (e) The figure shows the rise times for Combination 3. The average for the low and new-low levels of force lies within ± 3% of 15%. The average for the medium and new-medium lies within ± 10% of 25%. The average for the high force level lies within ± 30% of 70%. (f) The figure shows the settling times for combination 3. The average for the low and new-low levels of force lies within ± 12% of 72%. The average for the medium and new-medium lies within ± 10% of 25%. The average for the high force level lies within ± 8% of 60%.

Figure 4.

Subfigure (a) shows the rise times for Experiment 2, and Subfigure (b) shows the settling times of Experiment 2. In both subfigures the average rise time for the low force and the average rise time for the medium force across all six days lies within ± 5% of 20%. The average rise time for the high force lies within ± 8% of 22%. The average settling time for the low force lies within ± 10% of 88% The average settling time of the medium force lies within ± 20% of 64%. The average settling time of the high force lies within ± 10% of 65%.