Feasibility of using combined EMG and kinematic signals for prosthesis control: A simulation study using a virtual reality environment

Abstract

Transhumeral amputation has a significant effect on a person's independence and quality of life. Myoelectric prostheses have the potential to restore upper limb function, however their use is currently limited due to lack of intuitive and natural control of multiple degrees of freedom. The goal of this study was to evaluate a novel transhumeral prosthesis controller that uses a combination of kinematic and electromyographic (EMG) signals recorded from the person's proximal humerus. Specifically, we trained a time-delayed artificial neural network to predict elbow flexion/extension and forearm pronation/supination from six proximal EMG signals, and humeral angular velocity and linear acceleration. We evaluated this scheme with ten able-bodied subjects offline, as well as in a target-reaching task presented in an immersive virtual reality environment. The offline training had a target of 4° for flexion/extension and 8° for pronation/supination, which it easily exceeded (2.7° and 5.5° respectively). During online testing, all subjects completed the target-reaching task with path efficiency of 78% and minimal overshoot (1.5%). Thus, combining kinematic and muscle activity signals from the proximal humerus can provide adequate prosthesis control, and testing in a virtual reality environment can provide meaningful data on controller performance.

Keywords: Amputation; Artificial neural network; Control; Electromyography; Myoelectric; Prosthesis; Transhumeral.

Fig. 1

The two experimental phases of the study: the IMU-control and ANN-control phase. Shown are the EMG sensors around the circumference of the humerus (grey ovals), and three IMU (orange boxes, 1: thorax, 2: humerus, 3: forearm). Humeral angles are calculated by the combination of signals from the thorax and humerus IMU, and these are used to control the movement of the virtual humerus in the VRE. Similarly, elbow/forearm angles are calculated by the combination of signals from the humerus and forearm IMU, and these are used in the IMU-control phase to control the movement of the virtual forearm in the VRE. These are also used as output training signals for the ANN, while the input training signals are EMG and humerus angular velocity and linear acceleration, calculated from the humerus IMU. In the ANN-control phase, the ANN outputs are used to control the virtual forearm in the VRE instead of the IMU signals.

Fig. 2

The virtual reality environment, with a first-person view of a virtual person sitting at a desk. The participant controls the arm that is fully opaque, and tries to match the position and orientation of the less opaque (“target”) arm.

Fig. 3

An example of the input (panels A, B and C) and output (panel D) ANN training data. Panel A shows the six processed EMG signals, panel B shows the angular velocity of the humerus IMU, and panel C shows the linear acceleration of the humerus IMU. Panel D shows the elbow flexion/extension and forearm pronation/supination calculated based on the IMU on the humerus and forearm.

Fig. 4

The distribution of the values of the Index of Difficulty, for the targets used during the IMU-control phase (IMU) and the ANN-control phase (ANN).

Fig. 5

Summary of the movement metrics. Panel A shows the throughput for each subject, in the two experiment phases: IMU-control (IMU, dark bars), and ANN-control phase (ANN, light bars). Panel B shows the mean throughput for each experiment phase. Panel C shows the overshoot per subject, and Panel D shows the mean for each experiment phase. Similarly, Panel E shows the path efficiency per subject, and Panel F shows the mean for each experiment phase.

Fig. 6

Histogram of the time to target and the time remaining in each trial, after the last target was hit. Panel A shows the IMU-control phase, and Panel B shows the ANN-control phase.