Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Feb 28:17:1127338.
doi: 10.3389/fnbot.2023.1127338. eCollection 2023.

LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition

Wenli Zhang  1 , Tingsong Zhao  1 , Jianyi Zhang  2 , Yufei Wang  1
Affiliations

LST-EMG-Net: Long short-term transformer feature fusion network for sEMG gesture recognition

Wenli Zhang et al. Front Neurorobot. .

Abstract

With the development of signal analysis technology and artificial intelligence, surface electromyography (sEMG) signal gesture recognition is widely used in rehabilitation therapy, human-computer interaction, and other fields. Deep learning has gradually become the mainstream technology for gesture recognition. It is necessary to consider the characteristics of the surface EMG signal when constructing the deep learning model. The surface electromyography signal is an information carrier that can reflect neuromuscular activity. Under the same circumstances, a longer signal segment contains more information about muscle activity, and a shorter segment contains less information about muscle activity. Thus, signals with longer segments are suitable for recognizing gestures that mobilize complex muscle activity, and signals with shorter segments are suitable for recognizing gestures that mobilize simple muscle activity. However, current deep learning models usually extract features from single-length signal segments. This can easily cause a mismatch between the amount of information in the features and the information needed to recognize gestures, which is not conducive to improving the accuracy and stability of recognition. Therefore, in this article, we develop a long short-term transformer feature fusion network (referred to as LST-EMG-Net) that considers the differences in the timing lengths of EMG segments required for the recognition of different gestures. LST-EMG-Net imports multichannel sEMG datasets into a long short-term encoder. The encoder extracts the sEMG signals' long short-term features. Finally, we successfully fuse the features using a feature cross-attention module and output the gesture category. We evaluated LST-EMG-Net on multiple datasets based on sparse channels and high density. It reached 81.47, 88.24, and 98.95% accuracy on Ninapro DB2E2, DB5E3 partial gesture, and CapgMyo DB-c, respectively. Following the experiment, we demonstrated that LST-EMG-Net could increase the accuracy and stability of various gesture identification and recognition tasks better than existing networks.

Keywords: gesture recognition; human-computer interaction; multi-head attention; multi-scale features; sEMG signals; stroke rehabilitation.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

FIGURE 1
FIGURE 1
Types of gestures in the datasets used in this manuscript. (A) Ninapro DB2 exercise B dataset 17 gestures. (B) Ninapro DB5 exercise C dataset 18 gestures. (C) CapgMyo DB-c dataset 12 gestures.
FIGURE 2
FIGURE 2
Schematic diagram of time-delay enhancement module.
FIGURE 3
FIGURE 3
The proposed LST-EMG-Net structure. Among them, the sEmg channel attention, multi-head re-attention and the feature cross-attention module (yellow module) are the contribution points of this manuscript.
FIGURE 4
FIGURE 4
Feature cross-attention module.
See this image and copyright information in PMC

References

    1. Al-Saegh A., Dawwd S. A., Abdul-Jabbar J. M. (2021). Deep learning for motor imagery EEG-based classification: A review. Biomed. Signal Process. Control 63:102172. 10.1016/j.bspc.2020.102172 - DOI
    1. Alseed M. M., Tasoglu S. (2022). “Machine learning-enabled classification of forearm sEMG signals to control robotic hands prostheses,” in Proceedings of the 2022 innovations in intelligent systems and applications conference (ASYU) (Antalya: IEEE; ). 10.1109/ASYU56188.2022.9925273 - DOI
    1. Atzori M., Cognolato M., Müller H. (2016). Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands. Front. Neurorobot. 10:9. 10.3389/fnbot.2016.00009 - DOI - PMC - PubMed
    1. Atzori M., Gijsberts A., Castellini C., Caputo B., Hager A. G. M., Elsig S., et al. (2014a). Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 1 1–13. - PMC - PubMed
    1. Atzori M., Gijsberts A., Kuzborskij I., Elsig S., Hager A. G. M., Deriaz O., et al. (2014b). Characterization of a benchmark database for myoelectric movement classification. IEEE Trans. Neural Syst. Rehabil. Eng. 23 73–83. 10.1109/TNSRE.2014.2328495 - DOI - PubMed