Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 May 22;23(10):4980.
doi: 10.3390/s23104980.

WM-STGCN: A Novel Spatiotemporal Modeling Method for Parkinsonian Gait Recognition

Jieming Zhang  1 , Jongmin Lim  1 , Moon-Hyun Kim  2 , Sungwook Hur  1 , Tai-Myoung Chung  1
Affiliations

WM-STGCN: A Novel Spatiotemporal Modeling Method for Parkinsonian Gait Recognition

Jieming Zhang et al. Sensors (Basel). .

Abstract

Parkinson's disease (PD) is a neurodegenerative disorder that causes gait abnormalities. Early and accurate recognition of PD gait is crucial for effective treatment. Recently, deep learning techniques have shown promising results in PD gait analysis. However, most existing methods focus on severity estimation and frozen gait detection, while the recognition of Parkinsonian gait and normal gait from the forward video has not been reported. In this paper, we propose a novel spatiotemporal modeling method for PD gait recognition, named WM-STGCN, which utilizes a Weighted adjacency matrix with virtual connection and Multi-scale temporal convolution in a Spatiotemporal Graph Convolution Network. The weighted matrix enables different intensities to be assigned to different spatial features, including virtual connections, while the multi-scale temporal convolution helps to effectively capture the temporal features at different scales. Moreover, we employ various approaches to augment skeleton data. Experimental results show that our proposed method achieved the best accuracy of 87.1% and an F1 score of 92.85%, outperforming Long short-term memory (LSTM), K-nearest neighbors (KNN), Decision tree, AdaBoost, and ST-GCN models. Our proposed WM-STGCN provides an effective spatiotemporal modeling method for PD gait recognition that outperforms existing methods. It has the potential for clinical application in PD diagnosis and treatment.

Keywords: Parkinson’s disease; gait recognition; graph convolution network.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
The overall framework of the proposed method.
Figure 2
Figure 2
Experiment environment.
Figure 3
Figure 3
Walking trajectory and camera locations.
Figure 4
Figure 4
Data augmentation pipeline.
Figure 5
Figure 5
Temporal partition.
Figure 6
Figure 6
Joint coordinate space augmentation. (a) Joint coordinate translation; (b) Addition of Gaussian noise to the skeleton data.
Figure 7
Figure 7
One normal skeleton sequence example.
Figure 8
Figure 8
Spatiotemporal graph construction. (a) Spatial edges; (b) Temporal edges.
Figure 9
Figure 9
WM–STGCN framework. (a) The overall architecture of the proposed network; (b) The spatial module leverages the adjacency matrix to fuse features across joints; (c) The temporal module employs multi-scale temporal convolutions to capture temporal features.
Figure 10
Figure 10
Input data. (a) Adjacency matrix A; (b) Input feature map of the first GCN block.
Figure 11
Figure 11
(a) A graph of the input skeleton sequence; (b) The three submatrices.
Figure 12
Figure 12
(a) Virtual connection; (b) Diagram of the graph convolutional layer with weights.
Figure 13
Figure 13
Different parameters for weighted matrix.
Figure 14
Figure 14
Multi-scale temporal convolution network.
Figure 15
Figure 15
Performance of the several Gaussian noise augmentations.
Figure 16
Figure 16
Accuracy of the several Gaussian noise augmentations.
Figure 17
Figure 17
(a) Confusion matrix; (b) Loss function.
See this image and copyright information in PMC

References

    1. Mc Ardle R., Galna B., Donaghy P., Thomas A., Rochester L. Do Alzheimer’s and Lewy Body Disease Have Discrete Pathological Signatures of Gait? Alzheimer’s Dement. 2019;15:1367–1377. doi: 10.1016/j.jalz.2019.06.4953. - DOI - PubMed
    1. Beauchet O., Blumen H.M., Callisaya M.L., De Cock A.M., Kressig R.W., Srikanth V., Steinmetz J.P., Verghese J., Allali G. Spatiotemporal gait characteristics associated with cognitive impairment: A multicenter cross-sectional study, the intercontinental. Curr. Alzheimer Res. 2018;15:273–282. doi: 10.2174/1567205014666170725125621. - DOI - PubMed
    1. Mirelman A., Bonato P., Camicioli R., Ellis T.D., Giladi N., Hamilton J.L., Hass C.J., Hausdorff J.M., Pelosin E., Almeida Q.J. Gait impairments in Parkinson’s disease. Lancet Neurol. 2019;18:697–708. doi: 10.1016/S1474-4422(19)30044-4. - DOI - PubMed
    1. Goetz C.G., Tilley B.C., Shaftman S.R., Stebbins G.T., Fahn S., Martinez-Martin P., Poewe W., Sampaio C., Stern M.B., Dodel R., et al. Movement Disorder Society-Sponsored Revision of the Unified Parkinson’s Disease Rating Scale (MDS–UPDRS): Scale Presentation and Clinimetric Testing Results. Mov. Disord. 2008;23:2129–2170. doi: 10.1002/mds.22340. - DOI - PubMed
    1. Simpson G.M., Angus J.W.S. A Rating Scale for Extrapyramidal Side Effects. Acta Psychiatr. Scand. 1970;45:11–19. doi: 10.1111/j.1600-0447.1970.tb02066.x. - DOI - PubMed