Applying Machine Learning to Finger Movements Using Electromyography and Visualization in Opensim

Abstract

Electromyographic signals have been used with low-degree-of-freedom prostheses, and recently with multifunctional prostheses. Currently, they are also being used as inputs in the human-computer interface that controls interaction through hand gestures. Although there is a gap between academic publications on the control of an upper-limb prosthesis developed in laboratories and its service in the natural environment, there are attempts to achieve easier control using multiple muscle signals. This work contributes to this, using a database and biomechanical simulation software, both open access, to seek simplicity in the classifiers, anticipating their implementation in microcontrollers and their execution in real time. Fifteen predefined finger movements of the hand were identified using classic classifiers such as Bayes, linear and quadratic discriminant analysis. The idealized movements of the database were modeled with Opensim for visualization. Combinations of two preprocessing methods-the forward sequential selection method and the feature normalization method-were evaluated to increase the efficiency of these classifiers. The statistical methods of cross-validation, analysis of variance (ANOVA) and Duncan were used to validate the results. Furthermore, the classifier with the best recognition result was redesigned into a new feature space using the sparse matrix algorithm to improve it, and to determine which features can be eliminated without degrading the classification. The classifiers yielded promising results-the quadratic discriminant being the best, achieving an average recognition rate for each individual considered of 96.16%, and with 78.36% for the total sample group of the eight subjects, in an independent test dataset. The study ends with the visual analysis under Opensim of the classified movements, in which the usefulness of this simulation tool is appreciated by revealing the muscular participation, which can be useful during the design of a multifunctional prosthesis.

Keywords: biomechanical simulation; classification model; electromyography.

Figure 1

Electrode placement on the right forearm. (a) Anterior electrode positions; (b) posterior electrode positions; (c) muscle zones and electrodes placed on the cross-section of the forearm. (a,b) are reprinted/adapted with permission from Ref. [32], Copyright 2012, IEEE.

Figure 2

Experimentation conducted to determine the best option for creating a classification model considering recognition percentage and simplicity of the model. MU = single matrix (all eight subjects concatenated), MP = matrix per subject (eight matrices are formed). The number of circles defines the order of processing. The nomenclature (A–D) serves for Duncan’s significance test treatment identification that compares the output of each experiment.

Figure 3

General sequence of data manipulation: from the database files to the preparation of feature matrices, before the experiments being carried out.

Figure 4

Creation of the classification model. The decision and inference stages of machine learning can be observed in the creation of classification models.

Figure 5

Methodology for cross-validation. The original dataset is split several times to have partitions of data into subsets for model training and evaluation.

Figure 6

Creation of the classification model after selection of the best preprocessing from the conversion into a new space.

Figure 7

Screenshot of the software application displaying the degrees of freedom of the model used in Opensim.

Figure 8

Outliers maintained and eliminated. M = maintained, E = eliminated. The total data for the 8 subjects (sub #, subject number) in the 15 movements are displayed.

Figure 9

Plot: percentage of classification by selecting a smaller number of features after conversion to sparse matrices.

Figure 10

Hand gestures with perfect recognition. From left to right, the classes 10 (R_R), 1 (HC), 12 (T_L) and 8 (MRL).

Figure 11

Cycle from classification to generation of the movement file for Opensim.

Figure 12

Reproduction of the RL movement for 5 s in Opensim: (A) ideal movement; (B) total movement of repetition 1 for the movement of subject 6, which was the worst classified.