Estimation of Gait Mechanics Based on Simulated and Measured IMU Data Using an Artificial Neural Network

Abstract

Enhancement of activity is one major topic related to the aging society. Therefore, it is necessary to understand people's motion and identify possible risk factors during activity. Technology can be used to monitor motion patterns during daily life. Especially the use of artificial intelligence combined with wearable sensors can simplify measurement systems and might at some point replace the standard motion capturing using optical measurement technologies. Therefore, this study aims to analyze the estimation of 3D joint angles and joint moments of the lower limbs based on IMU data using a feedforward neural network. The dataset summarizes optical motion capture data of former studies and additional newly collected IMU data. Based on the optical data, the acceleration and angular rate of inertial sensors was simulated. The data was augmented by simulating different sensor positions and orientations. In this study, gait analysis was undertaken with 30 participants using a conventional motion capture set-up based on an optoelectronic system and force plates in parallel with a custom IMU system consisting of five sensors. A mean correlation coefficient of 0.85 for the joint angles and 0.95 for the joint moments was achieved. The RMSE for the joint angle prediction was smaller than 4.8° and the nRMSE for the joint moment prediction was below 13.0%. Especially in the sagittal motion plane good results could be achieved. As the measured dataset is rather small, data was synthesized to complement the measured data. The enlargement of the dataset improved the prediction of the joint angles. While size did not affect the joint moment prediction, the addition of noise to the dataset resulted in an improved prediction accuracy. This indicates that research on appropriate augmentation techniques for biomechanical data is useful to further improve machine learning applications.

Keywords: artificial neural networks; data simulation; inertial sensors; machine learning; motion analysis; wearable sensors.

Figure 1

Overview of the methods applied. To get the ground truth information on the joint angles and joint moments of the lower limbs, the gold standard approach using an optical motion capture system and force plates to collect the data is used. Based on this data, inverse dynamics simulations are undertaken to calculate the joint angles and joint moments. Using the proposed ML method, inertial data (angular rate ω and acceleration a) is simulated from the optical data and used as inputs for an artificial neural network. Based on the ground truth joint angles and moments, the network learns the connection between the input and output data. The method is validated using an IMU system based on five sensors that are placed consistently with the simulated data.

Figure 2

Marker set and sensor placement. The markers in the front are displayed in red, the ones at the back are displayed in blue. The green boxes display the IMU sensors.

Figure 3

Overview of the 5-fold cross-validation process. The dataset for the kinematics (A) and kinetics (B) differ and were treated separately.

Figure 4

Results of the synchronization based on the medio-lateral acceleration of the pelvis.

Figure 5

Overview of the leave-one-out validation process. Kinematics and kinetics were treated separately.

Figure 6

Root-mean-squared error between the measured and simulated data exemplarily displayed for the pelvis sensor. With an increasing gait velocity the simulation error increases. Some trials show outliers with larger errors during slow walking.

Figure 7

On the right, the distribution of the correlation coefficient for the kinematic (blue) and kinetic (red) model is displayed. Additionally, on top, the distribution of the RMSE for the kinematic and on the bottom the distribution of the nRMSE for the kinetic model can be found. The violin's width displays how much data is accumulated, while the height shows the range of the distribution. The horizontal line indicates the median value of the distribution.

Figure 8

Mean correlation coefficient for each joint and motion plane of each subject in the test set. On top, the results for the combined input data are displayed, while on bottom, the results of the model using measured data only are depicted. There are only small differences between both models, while there are distinct differences in the different motion planes and between subjects.

Figure 9

On the right, the distribution of the RMSE and the correlation coefficient for the kinematic data is displayed. On the left, the distribution of the nRMSE and the correlation coefficient for the kinetic model can be found. The results for the measured data as inputs is displayed in red, while the results for the combined data inputs are displayed in blue. The violin's width displays how much data is accumulated, while the height shows the range of the distribution. The horizontal line indicates the median value of the distribution.

Figure 10

Overview of the mean and standard deviation of the joint moments of the 23 subjects.

Figure 11

Overview of the mean and standard deviation of the joint angles of the 23 subjects.