Comparison of state-of-the-art deep learning architectures for detection of freezing of gait in Parkinson's disease

Abstract

Introduction: Freezing of gait (FOG) is one of the most debilitating motor symptoms experienced by patients with Parkinson's disease (PD). FOG detection is possible using acceleration data from wearable sensors, and a convolutional neural network (CNN) is often used to determine the presence of FOG epochs. We compared the performance of a standard CNN for the detection of FOG with two more complex networks, which are well suited for time series data, the MiniRocket and the InceptionTime.

Methods: We combined acceleration data of people with PD across four studies. The final data set was split into a training (80%) and hold-out test (20%) set. A fifth study was included as an unseen test set. The data were windowed (2 s) and five-fold cross-validation was applied. The CNN, MiniRocket, and InceptionTime models were evaluated using a receiver operating characteristic (ROC) curve and its area under the curve (AUC). Multiple sensor configurations were evaluated for the best model. The geometric mean was subsequently calculated to select the optimal threshold. The selected model and threshold were evaluated on the hold-out and unseen test set.

Results: A total of 70 participants (23.7 h, 9% FOG) were included in this study for training and testing, and in addition, 10 participants provided an unseen test set (2.4 h, 11% FOG). The CNN performed best (AUC = 0.86) in comparison to the InceptionTime (AUC = 0.82) and MiniRocket (AUC = 0.76) models. For the CNN, we found a similar performance for a seven-sensor configuration (lumbar, upper and lower legs and feet; AUC = 0.86), six-sensor configuration (upper and lower legs and feet; AUC = 0.87), and two-sensor configuration (lower legs; AUC = 0.86). The optimal threshold of 0.45 resulted in a sensitivity of 77% and a specificity of 58% for the hold-out set (AUC = 0.72), and a sensitivity of 85% and a specificity of 68% for the unseen test set (AUC = 0.90).

Conclusion: We confirmed that deep learning can be used to detect FOG in a large, heterogeneous dataset. The CNN model outperformed more complex networks. This model could be employed in future personalized interventions, with the ultimate goal of using automated FOG detection to trigger real-time cues to alleviate FOG in daily life.

Keywords: InceptionTime; MiniRocket; Parkinson’s disease; accelerometer; convolutional neural network; deep learning; freezing of gait; wearable sensors.

Figure 1

Overview of the CNN architecture.

Figure 2

Receiver operating characteristics of the classification models (mean area under the curve (AUC) of five folds ± standard deviation), results of the five-fold cross-validation. Results of: (A) the CNN, (B) the InceptionTime, and (C) the MiniRocket.

Figure 3

Receiver operating characteristics of the CNN with five different sensor configurations. Results are shown as the mean of the five-fold cross-validation. *Seven sensors: lumbar, upper legs, lower legs, and feet sensors. **Six sensors: upper legs, lower legs, and feet sensors.

Figure 4

Receiver operating characteristics of the lower legs CNN tested on the hold-out test set and unseen test set.