Development of a Robust, Simple, and Affordable Human Gait Analysis System Using Bottom-Up Pose Estimation With a Smartphone Camera

Abstract

Gait analysis is used in many fields such as Medical Diagnostics, Osteopathic medicine, Comparative and Sports-related biomechanics, etc. The most commonly used system for capturing gait is the advanced video camera-based passive marker system such as VICON. However, such systems are expensive, and reflective markers on subjects can be intrusive and time-consuming. Moreover, the setup of markers for certain rehabilitation patients, such as people with stroke or spinal cord injuries, could be difficult. Recently, some markerless systems were introduced to overcome the challenges of marker-based systems. However, current markerless systems have low accuracy and pose other challenges in gait analysis with people in long clothing, hiding the gait kinematics. The present work attempts to make an affordable, easy-to-use, accurate gait analysis system while addressing all the mentioned issues. The system in this study uses images from a video taken with a smartphone camera (800 × 600 pixels at an average rate of 30 frames per second). The system uses OpenPose, a 2D real-time multi-person keypoint detection technique. The system learns to associate body parts with individuals in the image using Convolutional Neural Networks (CNNs). This bottom-up system achieves high accuracy and real-time performance, regardless of the number of people in the image. The proposed system is called the "OpenPose based Markerless Gait Analysis System" (OMGait). Ankle, knee, and hip flexion/extension angle values were measured using OMGait in 16 healthy volunteers under different lighting and clothing conditions. The measured kinematic values were compared with a standard video camera based normative dataset and data from a markerless MS Kinect system. The mean absolute error value of the joint angles from the proposed system was less than 9⁰ for different lighting conditions and less than 11⁰ for different clothing conditions compared to the normative dataset. The proposed system is adequate in measuring the kinematic values of the ankle, knee, and hip. It also performs better than the markerless systems like MS Kinect that fail to measure the kinematics of ankle, knee, and hip joints under dark and bright light conditions and in subjects with long robe clothing.

Keywords: OpenPose; gait; kinematics; markerless system; smartphone.

FIGURE 1

(A) Schematic of the complete proposed simple system to measure kinematics of the gait, (B) location of the camera with respect to the subject walkway.

FIGURE 2

Shows how knee angle is calculated once the key points are located. The knee angle is calculated using the vector dot product. From the hip, knee, ankle coordinates obtained from the skeletal data two vectors are constructed. The first vector begins at the hip and ends at the knee while the second one begins at the knee and ends at the ankle. The knee angle (θ) is given by the following equation: $\mathrm{\theta }={\cos }^{-1}\frac{\overrightarrow{a}.\overrightarrow{b}}{|a||b|}$.

FIGURE 3

(A) Hip flexion/extension (B) Ankle dorsi/plantarflexion (C) Knee flexion/extension. Average values of all the subjects data using the proposed system shown in blue color and the average values from all the subjects from the gait database shown in red color.

FIGURE 4

Comparison of knee extension/flexion angle measured using OpenPose and Kinect.

FIGURE 5

(A) Ankle dorsi/plantarflexion measured under bright lighting of 9,800 lux. (B) Ankle dorsi/plantarflexion measured with participant in dark lighting of 50 lux. (C) Hip flex/extension measured with subject in bright lighting of 9,800 lux. (D) Hip flex/extension measured with participant in dark lighting of 50 lux. (E) Knee flex/extension measured with participant in bright lighting of 9,800 lux. (F) Knee flex/extension measured with participant in dark lighting of 50 lux.

FIGURE 6

Shows non-conventional clothing namely saree and dhoti.

FIGURE 7

(A) Ankle dorsi/plantarflexion. (B) Hip flex/extension. (C) Knee flex/extension measured with participant wearing non-conventional clothing and under average ambient lighting of 320 lux.

FIGURE 8

Showing (red arrows) variations at the start and end of gait cycles in our subjects.

FIGURE 9

Body pose estimation for extreme lighting conditions. (A) Original image. (B) Whitewashed image. (C) Darkened image. (D) BODY_25 pose estimation for whitewashed image. (E) BODY_25 pose estimation for darkened image.