A marker-less human motion analysis system for motion-based biomarker identification and quantification in knee disorders

Abstract

In recent years the healthcare industry has had increased difficulty seeing all low-risk patients, including but not limited to suspected osteoarthritis (OA) patients. To help address the increased waiting lists and shortages of staff, we propose a novel method of automated biomarker identification and quantification for the monitoring of treatment or disease progression through the analysis of clinical motion data captured from a standard RGB video camera. The proposed method allows for the measurement of biomechanics information and analysis of their clinical significance, in both a cheap and sensitive alternative to the traditional motion capture techniques. These methods and results validate the capabilities of standard RGB cameras in clinical environments to capture clinically relevant motion data. Our method focuses on generating 3D human shape and pose from 2D video data via adversarial training in a deep neural network with a self-attention mechanism to encode both spatial and temporal information. Biomarker identification using Principal Component Analysis (PCA) allows the production of representative features from motion data and uses these to generate a clinical report automatically. These new biomarkers can then be used to assess the success of treatment and track the progress of rehabilitation or to monitor the progression of the disease. These methods have been validated with a small clinical study, by administering a local anaesthetic to a small population with knee pain, this allows these new representative biomarkers to be validated as statistically significant ($p$-value $<0.05$). These significant biomarkers include the cumulative acceleration of elbow flexion/extension in a sit-to-stand, as well as the smoothness of the knee and elbow flexion/extension in both a squat and sit-to-stand.

Keywords: automated rehabilitation; biomarkers; biomechanics; human mesh recovery; human pose estimation; machine learning; patient monitoring.

Figure 1

Two examples of knee MRIs in the sagittal view from this study’s patients to highlight the severity of osteoarthritis, as shown by the lack of cartilage around the knee joint.

Figure 2

This represents the flow of the data from the source of the videos to the output of the statistical tests which allow the extraction of any significant data, this application has been applied to a clinical case study to examine the effectiveness of each technique when applied to intervention success.

Figure 3

Visual representation of the joint angles used in this study, showing both the sagittal and coronal joint angles and the location of the joint centres used in the calculations for each angle.

Figure 4

The most common features among the most representative biomarkers during the squat action.

Figure 5

The most common features among the most representative biomarkers during the sit-to-stand action.

Figure 6

Bland Altman plot showing the difference against the mean for each patient (for the smoothness of the maximum right knee flexion) during the sit-to-stand action. The variation around the mean shows the apparent differences before and after the injection.

Figure 7

Bland Altman plot showing the difference against the mean for each patient (for the smoothness of the maximum right knee flexion) during the squat action. The variation around the mean shows the apparent differences before and after the injection.

Figure 8

Box plot showing the median as well as the first and third quartile for percentage change for the entire clinical trial population from pre- to post-injection for the squat action projected on a logarithmic scale, showing the biomarkers identified to be statistically significant from the paired $t$-tests.

Figure 9

Box plot showing the median as well as the first and third quartile for percentage change for the entire clinical trial population from pre- to post-injection for the sit-to-stand action projected on a logarithmic scale, showing the biomarkers identified to be statistically significant from the paired $t$-tests.