Single-camera motion capture of finger joint mobility as a digital biomarker for disease activity in rheumatoid arthritis

Abstract

Objective: To investigate the association between hand motion tracking features obtained through computer vision from smartphone cameras and disease activity in patients with RA.

Methods: The PyPI package of MediaPipe (version 0.9.0.1) was used for key landmark detection. Finger joint angles were calculated in each frame using the normalized dot product of the vectors (equations). RA patients were instructed to perform a rapid repetition of five fist closures. Hand movements were captured using standard smartphone cameras. Kinetic features time to maximum flexion for MCP, PIP and DIP joints were correlated with RA disease activity and disability outcomes. Logistic regression was used to investigate associations of range of motion and kinetic features with 28-joint DAS (DAS28) low disease activity/remission.

Results: Our model showed promising performance in predicting low disease activity/remission in RA patients. Internal validation using 5-fold cross-validation on the training dataset (n = 81) yielded a mean accuracy of 0.72 (s.d. 0.09), specificity of 0.65 (s.d. 0.17), recall of 0.86 (s.d. 0.05) and area under the receiver operating characteristics curve (AUROC) of 0.80 (s.d. 0.09). External validation on the test dataset (n = 19) demonstrated improved performance with an accuracy of 0.84, specificity of 0.75, recall of 0.91 and AUROC of 0.89. Greater PIP and DIP joint angle changes, along with faster time to maximal flexion, were associated with lower disease activity. Significant correlations were observed between kinetic metrics and standard clinical measures, including DAS28, swollen joint count, tender joint count and HAQ Disability Index.

Conclusion: Single-camera motion capture of repeated fist closure may serve as an accessible digital biomarker for disease activity in RA.

Keywords: AI; RA; RPM; artificial intelligence; computer vision; digital biomarker; disease activity; finger joint mobility; hand motion tracking; kinetic features; remote patient monitoring; rheumatoid arthritis; smartphone camera.

Figure 1.

MeFISTO workflow showing (A) the key points schematically and (B) motion capture during fist closure in an exemplary RA patient

Figure 2.

Correlation plots for kinetic parameters and range of motion in the training dataset. Only significant correlations have been plotted. Green background: correlations plot for time for maximal flexion; cyan background: correlations plot for MCP degree change at maximal flexion; red background: correlations plot for PIP degree change at maximal flexion; orange background: correlation plot for HAQ-DI

Figure 3.

Model performance. (A) AUROC of the logistic regression model on the training dataset upon 5-fold cross-validation. (B) AUROC of the logistic regression model fitted with kinetic parameters on the test dataset