Physically Consistent Whole-Body Kinematics Assessment Based on an RGB-D Sensor. Application to Simple Rehabilitation Exercises

Abstract

This work proposes to improve the accuracy of joint angle estimates obtained from an RGB-D sensor. It is based on a constrained extended Kalman Filter that tracks inputted measured joint centers. Since the proposed approach uses a biomechanical model, it allows physically consistent constrained joint angles and constant segment lengths to be obtained. A practical method that is not sensor-specific for the optimal tuning of the extended Kalman filter covariance matrices is provided. It uses reference data obtained from a stereophotogrammetric system but it has to be tuned only once since it is task-specific only. The improvement of the optimal tuning over classical methods in setting the covariance matrices is shown with a statistical parametric mapping analysis. The proposed approach was tested with six healthy subjects who performed four rehabilitation tasks. The accuracy of joint angle estimates was assessed with a reference stereophotogrammetric system. Even if some joint angles, such as the internal/external rotations, were not well estimated, the proposed optimized algorithm reached a satisfactory average root mean square difference of 9.7 ∘ and a correlation coefficient of 0.8 for all joints. Our results show that an affordable RGB-D sensor can be used for simple in-home rehabilitation when using a constrained biomechanical model.

Keywords: extended Kalman filter and rehabilitation; markerless human motion analysis.

Figure 1

(Left) A 22 degree-of-freedom model of the locomotor system. ${\theta }_i$ is the ith joint angle corresponding to a rotation along the ith local Z axis and $L_j$ is the length of the segment j. (Right) Location of the retro-reflective markers (blue), used by the reference stereophotogrammetric system (SS), and of the Kinect sensor (K2S)-estimated joint center position (JCP) (green).

Figure 2

Tasks for measurement: (1) squat with lateral arm extensions, (2) stepping (left and right), (3) body tilt in sagittal plane and (4) body tilt in frontal plane.

Figure 3

Distribution of the difference between joint position centers estimated from the SS and from the K2S obtained for all recorded data along the x (blue), y (green) and z (yellow) axes. The grey distribution indicates the average one.

Figure 4

Representative comparison of joint trajectories estimated using the K2S and the proposed constrained extended Kalman filter (CEKF) (blue) and the SS and multibody kinetic optimization (MKO) (black) during the squat task. Flex/ext, int/ext and add/abd represent flexion/extension, internal/external rotation and adduction/abduction, respectively.

Figure 5

Representative evolution of the estimation of the segment lengths corresponding to the model described in Section 3.1. The segment lengths converge toward a constant fixed realistic value.

Figure 6

Representative results obtained with the paired t-test statistical parametric mapping (SPM). The top figures show descriptive statistics for each motion (Mean ±1 SD error cloud from between-subject variability) when the CEKF’s covariance matrices are optimized (green) and when classical methods are used to tune them (blue). Bottom figures show the most frequent inferences. The thick black line depicts the test statistic continuum or SPM{t}. The red horizontal dashed lines illustrate the critical t* based on $\alpha =0.05$. All areas outside the dashed red lines (grey) represent a p-value inferior to 5%. (a) is the angle ${\theta }_{14}$ for task 1, (b) the angle ${\theta }_4$ of task 1, (c) the angle ${\theta }_5$ of task 4, (d) the angle ${\theta }_5$ of task 2, and (e) is the angle ${\theta }_9$ of task 3