A multi-camera and multimodal dataset for posture and gait analysis

Abstract

Monitoring gait and posture while using assisting robotic devices is relevant to attain effective assistance and assess the user's progression throughout time. This work presents a multi-camera, multimodal, and detailed dataset involving 14 healthy participants walking with a wheeled robotic walker equipped with a pair of affordable cameras. Depth data were acquired at 30 fps and synchronized with inertial data from Xsens MTw Awinda sensors and kinematic data from the segments of the Xsens biomechanical model, acquired at 60 Hz. Participants walked with the robotic walker at 3 different gait speeds, across 3 different walking scenarios/paths at 3 different locations. In total, this dataset provides approximately 92 minutes of total recording time, which corresponds to nearly 166.000 samples of synchronized data. This dataset may contribute to the scientific research by allowing the development and evaluation of: (i) vision-based pose estimation algorithms, exploring classic or deep learning approaches; (ii) human detection and tracking algorithms; (iii) movement forecasting; and (iv) biomechanical analysis of gait/posture when using a rehabilitation device.

Fig. 1

Inertial sensors placement: (a) participant’s anterior view, and (b) participant’s posterior view. Note that each sensor was placed in the inner of the strap to better secure the sensors. Additionally, the sternum, head, and foot sensors were placed inside the frontal pocket of the Xsens suit, on the pocket of the headband and inside each shoe, respectively.

Fig. 2

Robotic walker used to collect this dataset, considering (a) the frontal view and (b) the rear view. Both upper and lower cameras are highlighted in (a) with a white box.

Fig. 3

Transformations to spatially align the Xsens skeleton data from a world referential (left) to the walker’s posture camera referential (right). Transformations are shown in 3D space along with the 3D point cloud for reference. Once the skeleton is aligned in 3D space, it is possible to project it to the cameras’ referential.

Fig. 4

Hierarchical folder structure of the database.

Fig. 5

Samples of processed data from a forward scenario (upper) and cornering scenario (lower): (a) Outside view of the acquisition setup taken from a smartphone; (b) and (d) Depth camera frame overlaid with aligned 2D skeleton; and (c) and (e) 3D point cloud overlaid with aligned 3D skeleton. Data from the upper and lower cameras are related through the extrinsic transformation and displayed together.