Evaluation of at-home physiotherapy

Abstract

An objective technological solution for tracking adherence to at-home shoulder physiotherapy is important for improving patient engagement and rehabilitation outcomes, but remains a significant challenge. The aim of this research was to evaluate performance of machine-learning (ML) methodologies for detecting and classifying inertial data collected during in-clinic and at-home shoulder physiotherapy exercise. A smartwatch was used to collect inertial data from 42 patients performing shoulder physiotherapy exercises for rotator cuff injuries in both in-clinic and at-home settings. A two-stage ML approach was used to detect out-of-distribution (OOD) data (to remove non-exercise data) and subsequently for classification of exercises. We evaluated the performance impact of grouping exercises by motion type, inclusion of non-exercise data for algorithm training, and a patient-specific approach to exercise classification. Algorithm performance was evaluated using both in-clinic and at-home data. The patient-specific approach with engineered features achieved the highest in-clinic performance for differentiating physiotherapy exercise from non-exercise activity (area under the receiver operating characteristic (AUROC) = 0.924). Including non-exercise data in algorithm training further improved classifier performance (random forest, AUROC = 0.985). The highest accuracy achieved for classifying individual in-clinic exercises was 0.903, using a patient-specific method with deep neural network model extracted features. Grouping exercises by motion type improved exercise classification. For at-home data, OOD detection yielded similar performance with the non-exercise data in the algorithm training (fully convolutional network AUROC = 0.919). Including non-exercise data in algorithm training improves detection of exercises. A patient-specific approach leveraging data from earlier patient-supervised sessions should be considered but is highly dependent on per-patient data quality.

Fig. 1

(System) data collection and machine-learning pipeline for identification and classification of at-home shoulder physiotherapy exercises. OOD, out-of-distribution.

Fig. 2

Patient-specific support method. In stage 1, Patient 0 is in the test split of the cross-validation fold. In stage 2, a feature extractor (e.g. fully convolutional network) is trained on the training split, while the last test fold session is used as ‘support’ data to train a K-Nearest Neighbour (KNN) algorithm on this more limited, but patient-specific, distribution. In stage 3, the data of the patient’s test session are transformed by the trained feature extractor and passed as input to the trained KNN algorithm to output the predictions of stage 4.

Fig. 3

Sample accelerometer inertial data of resisted triceps pull down (standing) in the labelled test set collected in clinic by Patient P3. Acc-x, Acc-y, and Acc-z indicate acceleration in x, y, and z axes of the accelerometer.

Fig. 4

Sample performance of out-of-distribution (OOD) prediction of at-home physiotherapy exercise activity on two records of accelerometer inertial data from patient P0 with a) patient-specific support method with engineered features (without proxy dataset) (area under the receiver operating characteristic (AUROC) 0.907), and b) random forest – fully convolutional network embeddings with OOD proxy dataset in the training set (AUROC 0.968). The misclassified exercise shown in a) exhibits very little motion relative to the other exercises in that record. Improvements from including proxy in training were similar for all algorithms.

Fig. 5

a) This record shows > five hours of data that appear to consist entirely of non-exercise, most of which is incorrectly predicted as exercise by the patient-specific method with engineered features without proxy (specificity 0.445). When the supervised labelled press-up record of b) is removed from the support set for this record, the resulting improvement is shown in c) (specificity 0.985). Note that it was not only this particular press-up record that caused this issue, but also any previous press-up exercises that the patient performed in clinic at earlier dates that were substituted in support.

Fig. 6

Examples of out-of-distribution detection and in-distribution classification on record from patients a) P1 and b) P4 with fully convolutional network supervised with proxy in training, grouped by simple motion category.