A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

Abstract

Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop person-specific prosthesis controllers that can predict a user's intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users' intentions; the timing of specific wrist and hand control functions; and users' opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details-spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the "limb position effect". We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.

Fig 1. RCNN-TL and LDA-baseline’s model training and testing.

The blue panel (A) illustrates the step that the General Participant Group performed (training routine that yielded RCNN-TL’s pre-trained model) while wearing an EMG and IMU armband. The yellow panels illustrate the steps that the Simulated Prosthesis (SP) Participant Group performed: (B) respective training routines that yielded RCNN-TL’s retrained model and LDA-Baseline’s trained model, and (C) subsequent controller testing using functional tasks, all while wearing an EMG and IMU armband plus a simulated prosthesis.

Fig 2. Myo armband and simulated prosthesis.

A) Myo armband on a participant’s forearm and B) simulated prosthesis on a participant’s forearm, with labels indicating the sleeve, two pieces of liner, hand brace, distal ring, cushions, wrist motor, and hand motor. Adapted from Williams et al. [26].

Fig 3. Architecture of RCNN-TL’s model: Sequence input layer; sequence folding layer; two blocks of 2D convolution, batch normalization, rectified linear unit (ReLU), and average pooling; one block of 2D convolution, batch normalization, and ReLU; sequence unfolding layer; flatten layer; long short-term memory (LSTM) layer; fully connected layer; softmax layer; and classification layer.

Adapted from Williams et al. [26].

Fig 4. Motion capture markers affixed to the simulated prosthesis.

The eight motion capture markers that remained attached to the hand are circled, and the three additional individual markers for the ski pose calibration are labelled.

Fig 5. Task setup for (A) Pasta and (B) RCRT Up and Down.

In panel (A), the 1^st, 2^nd, and 3^rd pasta box locations are labelled. The pasta box movement sequence is 1^st —>2^nd —>3^rd —>1^st locations. In panel (B), the 1^st, 2^nd, and 3^rd clothespin locations on the horizontal and vertical bars are labelled. The clothespin movement sequences in RCRT Up are horizontal 1^st —>vertical 1^st, horizontal 2^nd —>vertical 2^nd, and horizontal 3^rd —>vertical 3^rd locations. The clothespin movement sequence in RCRT Down follows the same order, but with each clothespin moved from vertical to horizontal locations.

Fig 6. Box plots indicating LDA-Baseline number of wrist rotation adjustments.

in each (A) RCRT Down Release and (B) RCRT Down Grasp of each task movement (Mvmt). Medians are indicated with thick lines, and interquartile ranges are indicated with boxes.

Fig 7. Box plots indicating user experience metrics results with RCNN-TL (orange) and LDA-Baseline (grey) for: (A) NASA-TLX, and (B) usability survey.

Medians are indicated with thick lines, interquartile ranges are indicated with boxes, and outliers are indicated with circles.