Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees

Abstract

Powered prosthetic legs are becoming a promising option for amputee patients. However, developing safe, robust, and intuitive control strategies for powered legs remains one of the greatest challenges. Although a variety of control strategies have been proposed, creating and fine-tuning the system parameters is time-intensive and complicated when more activities need to be restored. In this study, we developed a deep neural network (DNN) model that facilitates seamless and intuitive gait generation and transitions across five ambulation modes: level-ground walking, ascending/descending ramps, and ascending/descending stairs. The combination of latent and time sequence features generated the desired impedance parameters within the ambulation modes and allowed seamless transitions between ambulation modes. The model was applied to the open-source bionic leg and tested on unilateral transfemoral users. It achieved the overall coefficient of determination of 0.72 with the state machine-based impedance parameters in the offline testing session. In addition, users were able to perform in-laboratory ambulation modes with an overall success rate of 96% during the online testing session. The results indicate that the DNN model is a promising candidate for subject-independent and tuning-free prosthetic leg control for transfemoral amputees.

Keywords: ambulation modes; an open-source bionic leg; deep learning; impedance control; prosthetics.

Figure 1.

Architecture of the proposed DNN. The model takes a history of 18 mechanical sensor datapoints (see Table 1) as the inputs to obtain the single-time step impedance parameters. The latent network (a) extracts latent features (b). The latent features and history of three sensor data (weight, thigh angle, and shank angle) are used as the inputs of the main network (c) to extract time sequence features (d) and output impedance parameters. (e) The network modules represent the function of layers. The number in the brackets represents the number of units in the layers (e.g., Dense (N) represents the Dense layer with N units, and Load cell (Hist, N) represents N load cell sensor data points of the Hist-time step).

Figure 2.

System configuration. The Pyboard was connected with the OSL and the Android smartphone, respectively, by CAN and USB. The Pyboard acted as a bidirectional translation module between the smartphone and the OSL.

Figure 3.

Comparison of impedance parameters generated from the state machine (blue lines) and DNN (red lines) in offline tests. In general, the DNN made impedance parameters similar to the state machine for the user who was used to train the DNN model (i.e., TF2) and the new users (i.e., TF5 and TF6). The median across all impedance parameters for all ambulation modes were 0.77, 0.46, and 0.79 for TF2, TF5, and TF6, respectively; the median for all users was 0.72. All plots show 75th and 25th percentiles in lighter bands.

Figure 4.

RMSD of all five ambulation modes in offline tests. The users who were not used for the training (i.e., TF5 and TF6) showed a similar error level as the user who was used for training (i.e., TF2). Bar plots show the 25th, 50th (median), and 75th percentiles. Asterisk indicates  < .05.

Figure 5.

Gait trajectory and corresponding impedance parameters across five ambulation modes during online testing. The blue, red, and yellow lines represent TF2, TF3, and TF5, respectively. All plots show 75th and 25th percentiles in lighter bands.

Figure 6.

Visualization of latent features using t-SNE from the online test dataset. (a) Feature distributions for all ambulation modes. Features are clearly separated except for LGW and RA because their characteristics are similar. (b)–(f) present feature distributions of individual ambulation modes. The feature distributions have long and curved shapes because the features represent a continuous gait trajectory.

Figure 7.

Median gait trajectory (upper figures) and corresponding time sequence feature distribution (lower figures) during online testing. The blue, yellow, and red lines in the upper figures represent the normalized weight, ankle angle, and knee angle, respectively. Each color in the lower figures represents the activation level of each time sequence feature. The time sequence features were normalized with respect to the maximum and minimum. The features were sensitively changed according to the gait phase.

Figure 8.

Feature correlation with respect to ambulation modes and users. High correlation (minimum of 0.46 and median of 0.82) indicates that the time sequence features are sensitive to the gait phase, rather than ambulation mode. SD for TF2 was excluded because TF2 was unable to complete the mode.

Figure 9.

Gait comparison of offline (blue lines) and online data (red lines). All plots show 75th and 25th percentiles in lighter bands.

Figure 10.

Comparison of gait duration between offline data (blue lines) and online data (red lines) for five ambulation modes. In general, the DNN results in longer gait duration than the state machine. Bar plots show the 25th, 50th (median), and 75th percentiles. Asterisk indicates  < .05.