Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Jan 9:11:1335251.
doi: 10.3389/fbioe.2023.1335251. eCollection 2023.

Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video

Rui Hu #  1   2 , Yanan Diao #  1   2 , Yingchi Wang  1 , Gaoqiang Li  3 , Rong He  3 , Yunkun Ning  1 , Nan Lou  3 , Guanglin Li  1 , Guoru Zhao  1
Affiliations

Effective evaluation of HGcnMLP method for markerless 3D pose estimation of musculoskeletal diseases patients based on smartphone monocular video

Rui Hu et al. Front Bioeng Biotechnol. .

Abstract

Markerless pose estimation based on computer vision provides a simpler and cheaper alternative to human motion capture, with great potential for clinical diagnosis and remote rehabilitation assessment. Currently, the markerless 3D pose estimation is mainly based on multi-view technology, while the more promising single-view technology has defects such as low accuracy and reliability, which seriously limits clinical application. This study proposes a high-resolution graph convolutional multilayer perception (HGcnMLP) human 3D pose estimation framework for smartphone monocular videos and estimates 15 healthy adults and 12 patients with musculoskeletal disorders (sarcopenia and osteoarthritis) gait spatiotemporal, knee angle, and center-of-mass (COM) velocity parameters, etc., and compared with the VICON gold standard system. The results show that most of the calculated parameters have excellent reliability (VICON, ICC (2, k): 0.853-0.982; Phone, ICC (2, k): 0.839-0.975) and validity (Pearson r: 0.808-0.978, p<0.05). In addition, the proposed system can better evaluate human gait balance ability, and the K-means++ clustering algorithm can successfully distinguish patients into different recovery level groups. This study verifies the potential of a single smartphone video for 3D human pose estimation for rehabilitation auxiliary diagnosis and balance level recognition, and is an effective attempt at the clinical application of emerging computer vision technology. In the future, it is hoped that the corresponding smartphone program will be developed to provide a low-cost, effective, and simple new tool for remote monitoring and rehabilitation assessment of patients.

Keywords: high-resolution graph convolutional multilayer perception (HGcnMLP); markerless pose estimation; musculoskeletal disorders; rehabilitation assessment; smartphone monocular video.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

FIGURE 1
FIGURE 1
The flowchart and objectives of this research.
FIGURE 2
FIGURE 2
Experimental Setup, (A) Site setup, (B) Experimental site, (C) Sit-up test (Validation experiment), (D) Gait test (Validation experiment), (E) Markers (Front), (F) Markers (Back), (G) TUG test (Measurement experiment), and (H) Gait test (Measurement experiment).
FIGURE 3
FIGURE 3
The Flowchart of the Method includes Preprocessing, Extracting 2D key points, Extracting 3D key points, Feature extraction, and Data analysis.
FIGURE 4
FIGURE 4
Partial Parameter Results in the Validation Experiment:Vicon data (red) and Phone data (blue).
FIGURE 5
FIGURE 5
Distribution of Gait Spatiotemporal Parameter Results: Healthy adults (blue) and patients (red); (A) Left step lengths, Right step lengths, Left walking speed, and Right walking speed; (B) Left step period, Right step period, Left swing time, Right swing time, Left support time, and Right support time.
FIGURE 6
FIGURE 6
Joint Angle Results: Healthy adults (subject 1–13) and patients (subject 14–25). (A) Range of Knee Angles, (B) Box Plot of Knee Angular Velocity.
FIGURE 7
FIGURE 7
COM Results: Healthy adults (blue) and patients (red).
FIGURE 8
FIGURE 8
Clustering Results:Healthy adults (circles), patients (squares).
See this image and copyright information in PMC

References

    1. Åberg A. C., Olsson F., Åhman H. B., Tarassova O., Arndt A., Giedraitis V., et al. (2021). Extraction of gait parameters from marker-free video recordings of timed up-and-go tests: validity, inter-and intra-rater reliability. Gait Posture 90, 489–495. 10.1016/j.gaitpost.2021.08.004 - DOI - PubMed
    1. Aleixo P., Vaz Patto J., Cardoso A., Moreira H., Abrantes J. (2019). Ankle kinematics and kinetics during gait in healthy and rheumatoid arthritis post-menopausal women. Somatosens. Mot. Res. 36, 171–178. 10.1080/08990220.2019.1634536 - DOI - PubMed
    1. Aoyagi Y., Yamada S., Ueda S., Iseki C., Kondo T., Mori K., et al. (2022). Development of smartphone application for markerless three-dimensional motion capture based on deep learning model. Sensors 22, 5282. 10.3390/s22145282 - DOI - PMC - PubMed
    1. Arthur D., Vassilvitskii S. (2007). “K-means++ the advantages of careful seeding,” in Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, New Orleans, Louisiana, USA, January 7-9, 2007, 1027–1035.
    1. Avogaro A., Cunico F., Rosenhahn B., Setti F. (2023). Markerless human pose estimation for biomedical applications: a survey. arXiv preprint arXiv:2308.00519 .

LinkOut - more resources