Dual Stream Long Short-Term Memory Feature Fusion Classifier for Surface Electromyography Gesture Recognition

Abstract

Gesture recognition using electromyography (EMG) signals has prevailed recently in the field of human-computer interactions for controlling intelligent prosthetics. Currently, machine learning and deep learning are the two most commonly employed methods for classifying hand gestures. Despite traditional machine learning methods already achieving impressive performance, it is still a huge amount of work to carry out feature extraction manually. The existing deep learning methods utilize complex neural network architectures to achieve higher accuracy, which will suffer from overfitting, insufficient adaptability, and low recognition accuracy. To improve the existing phenomenon, a novel lightweight model named dual stream LSTM feature fusion classifier is proposed based on the concatenation of five time-domain features of EMG signals and raw data, which are both processed with one-dimensional convolutional neural networks and LSTM layers to carry out the classification. The proposed method can effectively capture global features of EMG signals using a simple architecture, which means less computational cost. An experiment is conducted on a public DB1 dataset with 52 gestures, and each of the 27 subjects repeats every gesture 10 times. The accuracy rate achieved by the model is 89.66%, which is comparable to that achieved by more complex deep learning neural networks, and the inference time for each gesture is 87.6 ms, which can also be implied in a real-time control system. The proposed model is validated using a subject-wise experiment on 10 out of the 40 subjects in the DB2 dataset, achieving a mean accuracy of 91.74%. This is illustrated by its ability to fuse time-domain features and raw data to extract more effective information from the sEMG signal and select an appropriate, efficient, lightweight network to enhance the recognition results.

Keywords: deep learning; dual stream LSTM; feature fusion; gesture recognition.

Figure 1

Illustraion of 1D CNN using an example of one channel signal.

Figure 2

LSTM architecture diagram.

Figure 3

HGR diagram depicts steps included in the proposed dual stream LSTM feature fusion architecture (Data acquisition picture from “Atzori 2014 [28]”).

Figure 4

Framework of the dual stream model based on LSTM and CNN architecture.

Figure 5

The block diagram of dual stream model.

Figure 6

The 52 hand movements in the NinaPro DB1 Dataset [28].

Figure 7

Sliding window segmentation of the sEMG data.

Figure 8

Comparing accuracy across models A, B, and C.

Figure 9

Comparing loss across models A, B, and C.

Figure 10

The block diagram without the dual stream model.

Figure 11

Confusion matrix for dual stream feature fusion classifier. (Diagonal data: 0: 91.43, 1: 97.50, 2: 93.02, 3: 93.75, 4: 100, 5: 97.37, 6: 90.91, 7: 93.33, 8: 79.49, 9: 88.89, 10: 80.00, 11: 85.11, 12: 94.87, 13: 88.46, 14: 95.83, 15: 91.43, 16: 83.33, 17: 84.85, 18: 92.50, 19: 94.59, 20: 100, 21: 94.74, 22: 91.18, 23: 92.68, 24: 97.62, 25: 100, 26: 91.67, 27: 97.14, 28: 100.00, 29: 84.44, 30: 85.37, 31: 75.76, 32: 97.92, 33: 91.18, 34: 80.95, 35: 74.07, 36: 87.50, 37: 86.67, 38: 93.55, 39: 80.65, 40: 80.49, 41: 86.27, 42: 86.21, 43: 76.19, 44: 75.76, 45: 81.58, 46: 86.21, 47: 83.33, 48: 97.78, 49: 86.11, 50: 91.43, and 51: 95.45).

Figure 12

Validation accuracy comparison of models A, B, and C.

Figure 13

Validation accuracy of 10 subjects from DB2 dataset.