Using expert-cited features to detect leg dystonia in cerebral palsy

Abstract

Objectives: Leg dystonia in cerebral palsy (CP) is debilitating but remains underdiagnosed. Routine clinical evaluation has only 12% accuracy for leg dystonia diagnosis compared to gold-standard expert consensus assessment. We determined whether expert-cited leg dystonia features could be quantified to train machine learning (ML) models to detect leg dystonia in videos of children with CP.

Methods: Eight pediatric movement disorders physicians assessed 298 videos of children with CP performing a seated task at two CP centers. We extracted leg dystonia features cited by these experts during consensus-building discussions, quantified these features in videos, used these quantifications to train 4664 ML models on 163 videos from one center, and tested the best performing models on a separate set of 135 videos from both centers.

Results: We identified 69 quantifiable features corresponding to 12 expert-cited leg dystonia features. ML models trained using these quantifications achieved 88% sensitivity, 74% specificity, 82% positive predictive value, 84% negative predictive value, and 82% accuracy for identifying leg dystonia across both centers. Of the 25 features contributing to the best performing ML models, 17 (68%) quantified leg movement variability. We used these ML models to develop DxTonia, open-source software that identifies leg dystonia in videos of children with CP.

Interpretation: DxTonia primarily leverages detection of leg movement variability to achieve 82% accuracy in identifying leg dystonia in children with CP, a significant improvement over routine clinical diagnostic accuracy of 12%. Observing or quantifying leg movement variability during a seated task can facilitate leg dystonia detection in CP.

Figure 1.. Training, validation, and testing of machine-learning diagnostic model for leg dystonia.

We divided the full video dataset five times into different subsets for training (2/3 of the dataset), validation (1/6 of the dataset), and testing (1/6 of the dataset) to generate five different dataset groupings. We then conducted training and validation five times within each of the dataset groupings, using different subsets for training and validation each time.

Figure 2.

Frequencies of quantifiable movements cited by experts when diagnosing dystonia.

Figure 3.. Comparison of the mean importance ranking of variance, minimum, and maximum features.

Features were ranked from 1 (highest importance) to 69 (lowest importance). The mean feature importance ranking for variance features was significantly better than the mean rankings for minimum and maximum features (one-way ANOVA, *p < 0.005). Whiskers indicate the interquartile range, the upper and lower edges of the boxes represent the 75th and the 25th percentile of the distribution, respectively, and the red middle line represents the median of the distribution.

Figure 4.. The mean NPV and specificity of ML models when trained on different numbers of top-ranked features.

The black points denote the best performing model for each of the five dataset groupings of the full dataset. The x-axis represents the number of features used for training and validation the modles. The y-axis represents the mean validation NPV and specificity of the best-performing ML model (out of 4664 models) when trained on the selected number of features.