Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jun 4;24(11):3631.
doi: 10.3390/s24113631.

Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition

Kexin Zhang  1 , Francisco J Badesa  1 , Yinlong Liu  2 , Manuel Ferre Pérez  1
Affiliations

Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition

Kexin Zhang et al. Sensors (Basel). .

Abstract

Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.

Keywords: deep learning; dual stream LSTM; feature fusion; gesture recognition.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

Figure 1
Figure 1
Illustraion of 1D CNN using an example of one channel signal.
Figure 2
Figure 2
LSTM architecture diagram.
Figure 3
Figure 3
HGR diagram depicts steps included in the proposed dual stream LSTM feature fusion architecture (Data acquisition picture from “Atzori 2014 [28]”).
Figure 4
Figure 4
Framework of the dual stream model based on LSTM and CNN architecture.
Figure 5
Figure 5
The block diagram of dual stream model.
Figure 6
Figure 6
The 52 hand movements in the NinaPro DB1 Dataset [28].
Figure 7
Figure 7
Sliding window segmentation of the sEMG data.
Figure 8
Figure 8
Comparing accuracy across models A, B, and C.
Figure 9
Figure 9
Comparing loss across models A, B, and C.
Figure 10
Figure 10
The block diagram without the dual stream model.
Figure 11
Figure 11
Confusion matrix for dual stream feature fusion classifier. (Diagonal data: 0: 91.43, 1: 97.50, 2: 93.02, 3: 93.75, 4: 100, 5: 97.37, 6: 90.91, 7: 93.33, 8: 79.49, 9: 88.89, 10: 80.00, 11: 85.11, 12: 94.87, 13: 88.46, 14: 95.83, 15: 91.43, 16: 83.33, 17: 84.85, 18: 92.50, 19: 94.59, 20: 100, 21: 94.74, 22: 91.18, 23: 92.68, 24: 97.62, 25: 100, 26: 91.67, 27: 97.14, 28: 100.00, 29: 84.44, 30: 85.37, 31: 75.76, 32: 97.92, 33: 91.18, 34: 80.95, 35: 74.07, 36: 87.50, 37: 86.67, 38: 93.55, 39: 80.65, 40: 80.49, 41: 86.27, 42: 86.21, 43: 76.19, 44: 75.76, 45: 81.58, 46: 86.21, 47: 83.33, 48: 97.78, 49: 86.11, 50: 91.43, and 51: 95.45).
Figure 12
Figure 12
Validation accuracy comparison of models A, B, and C.
Figure 13
Figure 13
Validation accuracy of 10 subjects from DB2 dataset.
See this image and copyright information in PMC

References

    1. Pratt J., Krupp B., Morse C., Collins S. The RoboKnee: An exoskeleton for enhancing strength and endurance during walking; Proceedings of the IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA ’04. 2004; New Orleans, LA, USA. 26 April–1 May 2004; pp. 2430–2435. - DOI
    1. Borzelli D., Burdet E., Pastorelli S., D’Avella A., Gastaldi L. Identification of the best strategy to command variable stiffness using electromyographic signals. J. Neural Eng. 2020;17:016058. doi: 10.1088/1741-2552/ab6d88. - DOI - PubMed
    1. Farina D., Negro F. Accessing the Neural Drive to Muscle and Translation to Neurorehabilitation Technologies. IEEE Rev. Biomed. Eng. 2012;5:3–14. doi: 10.1109/RBME.2012.2183586. - DOI - PubMed
    1. Fu R., Zhang B., Liang H., Wang S., Wang Y., Li Z. Gesture recognition of sEMG signal based on GASF-LDA feature enhancement and adaptive ABC optimized SVM. Biomed. Signal Process. Control. 2023;85:105104. doi: 10.1016/j.bspc.2023.105104. - DOI
    1. Yu Z., Zhao J., Wang Y., Linglong H., Wang S. Surface EMG-Based Instantaneous Hand Gesture Recognition Using Convolutional Neural Network with the Transfer Learning Method. Sensors. 2021;21:2540. doi: 10.3390/s21072540. - DOI - PMC - PubMed

LinkOut - more resources