Novel AI driven approach to classify infant motor functions

Abstract

The past decade has evinced a boom of computer-based approaches to aid movement assessment in early infancy. Increasing interests have been dedicated to develop AI driven approaches to complement the classic Prechtl general movements assessment (GMA). This study proposes a novel machine learning algorithm to detect an age-specific movement pattern, the fidgety movements (FMs), in a prospectively collected sample of typically developing infants. Participants were recorded using a passive, single camera RGB video stream. The dataset of 2800 five-second snippets was annotated by two well-trained and experienced GMA assessors, with excellent inter- and intra-rater reliabilities. Using OpenPose, the infant full pose was recovered from the video stream in the form of a 25-points skeleton. This skeleton was used as input vector for a shallow multilayer neural network (SMNN). An ablation study was performed to justify the network's architecture and hyperparameters. We show for the first time that the SMNN is sufficient to discriminate fidgety from non-fidgety movements in a sample of age-specific typical movements with a classification accuracy of 88%. The computer-based solutions will complement original GMA to consistently perform accurate and efficient screening and diagnosis that may become universally accessible in daily clinical practice in the future.

Figure 1

Overview of the algorithm’s process pipeline.

Figure 2

An overview of the feature extraction and classification procedure. One snippet has a length of $250\mathrm{frames}$. $N_{\mathit{stack}}=52\mathrm{frames}$ are concatenated to one input vector—i.e. the (x,  y)-values of $52\mathrm{frames}$ frames are stacked together to form one input vector. The offset between two input vectors is $N_{\mathit{slide}}=12\mathrm{frames}$, resulting in a sliding window approach. Each input vector is classified independently. The final decision is made based on uniform majority vote.

Figure 3

(a) An example frame with 25-point skeleton overlay and (b) a schematic diagram of the SMNN 5 network architecture.

Figure 4

Comparison of SMNN architectures (Table 3) on classification accuracy. Mean classification accuracy obtained on five cross-validation test sets (Table 2) are shown for each model. Error bars denote confidence intervals of mean (95%). Mean difference of SMMN 5 and SMMN 4 is statistically significant ($t-test$, $p=0.0381$). Differences of all other means are not statistically significant ($t-test$, $p>0.05$ for all other pairs).

Figure 5

Running time comparison for neural network architectures SMNN 1–9. Mean training and inference frequency (number of samples per second) are shown for each model. Error bars denote standard deviation (SD).

Figure 6

Results of hyperparameter tuning obtained with SMNN 5. (a) Batch size $N_{\mathit{batch}}$, (b) number of frames per input vector $N_{\mathit{stack}}$, and (c) offset between two consecutive input vectors $N_{\mathit{slide}}$. Left: TPR vs. FPR scores for hyperparameter tuning (the value closest to the TPR = 1 and FPR = 0 corresponds to the best perfromance). Middle: distance $d=\sqrt{{\left(1,-,\mathrm{TPR}\right)}^2+{\left(\mathrm{FPR}\right)}^2}$ with respect to TPR = 1 and FPR = 0 (the lowest value corresponds to the best performance). Right: classification accuracy (the highest value corresponds to the best performance). Red dot denotes the parameter with the best performance.