Pose2Sim: An End-to-End Workflow for 3D Markerless Sports Kinematics-Part 2: Accuracy

Abstract

Two-dimensional deep-learning pose estimation algorithms can suffer from biases in joint pose localizations, which are reflected in triangulated coordinates, and then in 3D joint angle estimation. Pose2Sim, our robust markerless kinematics workflow, comes with a physically consistent OpenSim skeletal model, meant to mitigate these errors. Its accuracy was concurrently validated against a reference marker-based method. Lower-limb joint angles were estimated over three tasks (walking, running, and cycling) performed multiple times by one participant. When averaged over all joint angles, the coefficient of multiple correlation (CMC) remained above 0.9 in the sagittal plane, except for the hip in running, which suffered from a systematic 15° offset (CMC = 0.65), and for the ankle in cycling, which was partially occluded (CMC = 0.75). When averaged over all joint angles and all degrees of freedom, mean errors were 3.0°, 4.1°, and 4.0°, in walking, running, and cycling, respectively; and range of motion errors were 2.7°, 2.3°, and 4.3°, respectively. Given the magnitude of error traditionally reported in joint angles computed from a marker-based optoelectronic system, Pose2Sim is deemed accurate enough for the analysis of lower-body kinematics in walking, cycling, and running.

Keywords: OpenPose; OpenSim; accuracy; computer vision; concurrent validity; deep learning; kinematics; markerless motion capture; sports performance analysis.

Figure A1

Pose2Sim (cyan) and marker-based (black) sacro-lumbar and upper-body joint angles for the walking task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean error (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics.

Figure A2

Bland–Altman analysis of sacro-lumbar and upper-body joint angle errors for the walking task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines.

Figure A3

Pose2Sim (cyan) and marker-based (black) lower-body joint angles for the running task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean errors (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics.

Figure A4

Bland–Altman analysis of lower-body joint angle errors for the running task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines.

Figure A5

Pose2Sim (cyan) and marker-based (black) sacro-lumbar and upper-body joint angles for the running task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean error (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics.

Figure A6

Bland–Altman analysis of sacro-lumbar and upper-body joint angle errors for the running task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines.

Figure A7

Pose2Sim (cyan) and marker-based (black) lower-body joint angles for the cycling task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean errors (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics.

Figure A8

Bland–Altman analysis of lower-body joint angle errors for the cycling task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines.

Figure A9

Pose2Sim (cyan) and marker-based (black) sacro-lumbar and upper-body joint angles for the cycling task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean errors (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics.

Figure A10

Bland–Altman analysis of sacro-lumbar and upper-body joint angle errors for the cycling task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines.

Figure 1

Triangulated anatomical markers and clusters (dark green), calculated joint centers (light green), and OpenPose BODY_25B keypoints (pink) on a textured mesh. OpenPose’s eyes and ears keypoints were excluded [25]. Mesh opacity was set to 0.5 in order to make all points visible. This view made it possible to precisely place OpenPose triangulated keypoints on the OpenSim model.

Figure 2

Participant’s 3D textured meshes were extracted using 68 video cameras in the studio, and then placed in a virtual environment. The scene was then filmed from 8 virtual cameras.

Figure 3

Pose2Sim full pipeline: (1) OpenPose 2D joint detection; (2i) camera calibration; (2ii–iv) tracking the person of interest, triangulating his coordinates, and filtering them; (3) constraining the 3D coordinates to a physically consistent OpenSim skeletal model.

Figure 4

Pose2Sim (cyan) and marker-based (black) lower-body joint angles for the walking task. Coefficient of multiple correlation (CMC) is indicated, and broken down into, respectively, Pearson’s coefficient (r) for correlation assessment, range of motion errors (ROM_err) for gain, and overall mean errors (Mean_err) for offset. Mean error and standard deviations are also represented at the bottom of the graphics. See Appendix A for running and cycling results, and for sacro-lumbar and upper-body results.

Figure 5

Bland–Altman analysis of lower-body joint angle errors for the walking task. Mean bias is represented as a horizontal solid, bold line, and 95% limits of agreement are represented as dotted lines. See Appendix A for running and cycling results, and for sacro-lumbar and upper-body results.