The Role of Surface Electromyography in Data Fusion with Inertial Sensors to Enhance Locomotion Recognition and Prediction

Abstract

Locomotion recognition and prediction is essential for real-time human-machine interactive control. The integration of electromyography (EMG) with mechanical sensors could improve the performance of locomotion recognition. However, the potential of EMG in motion prediction is rarely discussed. This paper firstly investigated the effect of surface EMG on the prediction of locomotion while integrated with inertial data. We collected EMG signals of lower limb muscle groups and linear acceleration data of lower limb segments from ten healthy participants in seven locomotion activities. Classification models were built based on four machine learning methods-support vector machine (SVM), k-nearest neighbor (KNN), artificial neural network (ANN), and linear discriminant analysis (LDA)-where a major vote strategy and a content constraint rule were utilized for improving the online performance of the classification decision. We compared four classifiers and further investigated the effect of data fusion on the online locomotion classification. The results showed that the SVM model with a sliding window size of 80 ms achieved the best recognition performance. The fusion of EMG signals does not only improve the recognition accuracy of steady-state locomotion activity from 90% (using acceleration data only) to 98% (using data fusion) but also enables the prediction of the next steady locomotion (∼370 ms). The study demonstrates that the employment of EMG in locomotion recognition could enhance online prediction performance.

Keywords: data fusion; inertial sensor; locomotion prediction; locomotion recognition; machine learning; multimodal sensing; surface electromyography.

Figure 1

Experiment setup for data collection. The participant wore a set of sensors and retroreflective markers. EMG electrodes were placed on 14 muscles—rectus femoris (RF), vastus lateralis (VL), biceps femoris (BF), semitendinosus (Sem), tibialis anterior (TA), medial gastrocnemius (MG), and lateral gastrocnemius (LG)—of both legs. Motion sensors were placed on the pelvis, thigh, shank, and foot. A Plug-in-Gait (PiG) marker set was used as a reference for identifying the locomotion activities and gait cycles.

Figure 2

Schematic illustration of the experimental protocol. (a) Sit (S)–stand (ST)–level walk (LW)–stand (ST); (b) stand (ST)–level walk (LW)–stair ascent (SA)/stair descent (SD)–level walk (LW)–stand (ST); (c) stand (ST)–level walk (LW)–ramp ascent (RA)/ramp descent (RD)–level walk (LW)–stand (ST); (d) photos of one participant who performed the locomotion transition with her right leg.

Figure 3

Flowchart of the offline training and online test based on SVM, KNN, LDA, and ANN, respectively.

Figure 4

Effects of sliding window sizes and classifiers on: (a) the classification accuracy, (b) the predictive accuracy, and (c) the response time. The results were averaged over 10 participants. The shadows represent the +/– standard deviation (SD).

Figure 5

Confusion matrix for the steady-state locomotion recognition of seven locomotion modes for the four classifiers: (a) SVM; (b) KNN; (c) LDA; (d) ANN. The results in the confusion matrices were averaged over ten participants. The ST, S, LW, SA, SD, RA, and RD denote standing, sitting, level walking, stair ascent, stair descent, ramp ascent, and ramp descent, respectively.

Figure 6

Comparison of the steady-state locomotion recognition performance using sEMG, linear acceleration, and data fusion. * and ** demonstrate a statistically significant difference (t-test, *: p < 0:05, **: p < 0:001).

Figure 7

Comparison of the prediction performance using sEMG, acceleration, and sensor fusion signals. ST, S, LW, SA, SD, RA, and RD denote standing, sitting, level walking, stair ascent, stair descent, ramp ascent, and ramp descent, respectively. * and ** demonstrate a statistically significant difference (t-test, *: p < 0:05, **: p < 0:001).