Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Jun 5;4(1):24.
doi: 10.1186/s40798-018-0139-y.

A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System

Steffi L Colyer  1   2 , Murray Evans  1   3 , Darren P Cosker  1   3 , Aki I T Salo  4   5
Affiliations
Review

A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System

Steffi L Colyer et al. Sports Med Open. .

Abstract

Background: The study of human movement within sports biomechanics and rehabilitation settings has made considerable progress over recent decades. However, developing a motion analysis system that collects accurate kinematic data in a timely, unobtrusive and externally valid manner remains an open challenge.

Main body: This narrative review considers the evolution of methods for extracting kinematic information from images, observing how technology has progressed from laborious manual approaches to optoelectronic marker-based systems. The motion analysis systems which are currently most widely used in sports biomechanics and rehabilitation do not allow kinematic data to be collected automatically without the attachment of markers, controlled conditions and/or extensive processing times. These limitations can obstruct the routine use of motion capture in normal training or rehabilitation environments, and there is a clear desire for the development of automatic markerless systems. Such technology is emerging, often driven by the needs of the entertainment industry, and utilising many of the latest trends in computer vision and machine learning. However, the accuracy and practicality of these systems has yet to be fully scrutinised, meaning such markerless systems are not currently in widespread use within biomechanics.

Conclusions: This review aims to introduce the key state-of-the-art in markerless motion capture research from computer vision that is likely to have a future impact in biomechanics, while considering the challenges with accuracy and robustness that are yet to be addressed.

Keywords: Automatic analysis; Body model; Cameras; Discriminative approaches; Gait; Generative algorithms; Motion capture; Rehabilitation; Sports biomechanics; Technique.

PubMed Disclaimer

Conflict of interest statement

Authors’ Information

SC has a PhD and is a Post-Doctoral Research Associate in the CAMERA project (see funding above) with expertise in analysing athletes’ technique. ME has a PhD and is a Post-Doctoral Research Associate in the CAMERA project with expertise in computer vision with time spent in the computer industry before joining back to academia. DC has a PhD and is a Professor in Computer Science specialising in computer vision. He is the principal investigator and the director of the CAMERA project. AS has a PhD and is a Reader (Associate Professor) in Sports Biomechanics with expertise in analysing athletes’ technique. He is a co-investigator in the CAMERA project.

Ethics Approval and Consent to Participate

Not applicable

Competing Interests

The authors Steffi L. Colyer, Murray Evans, Darren P. Cosker and Aki I. T. Salo declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

Fig. 1
Fig. 1
General structure of a markerless motion capture whether using generative (green) or discriminative (orange) algorithms
Fig. 2
Fig. 2
Example of a depth map. Brighter pixels are further away from the camera. Black pixels are either too far away or on objects that do not reflect near infrared light
Fig. 3
Fig. 3
Example of a poseable skeleton model. “Bones” of a pre-specified length are connected at joints, and rotation of the bones around these joints allows the skeleton to be posed. The skeleton model is commonly fit to both marker-based motion capture data and computer vision-based markerless systems
Fig. 4
Fig. 4
Sum of Gaussian body model from Stoll [75]. A skeleton (left) forms the foundation of the model, providing limb-lengths and body pose. The body is given volume and appearance information through the use of 3D Spatial Gaussians arranged along the skeleton (represented by spheres). The resulting information allows the model to be fit to image data
Fig. 5
Fig. 5
Skinned Multi-Person Linear Model (SMPL) [79] body model. This model does not have an explicit skeleton. Instead, the surface of a person is represented by a mesh of triangles. A set of parameters (learnt through regression) allows the shape of the model to be changed from a neutral mean (left) to a fatter (middle) or thinner, taller, or other body shape. Once shaped, the centres of joints are inferred from the neutrally posed mesh, and then the mesh can be rotated around these joints to produce a posed body (right)
Fig. 6
Fig. 6
Silhouette on the right from chroma keying the image on the left. When seen as only a silhouette, it is not possible to infer if the mannequin is facing towards or away from the camera
Fig. 7
Fig. 7
The generation of a visual hull, which is a type of 3D reconstruction of an object viewed from multiple cameras. Top row: images of an object are captured as 2D images from multiple directions. Middle row: these images are processed to produce silhouette images for each camera. Bottom left: the silhouettes are back-projected from each camera, resulting in cone-like regions of space. Bottom right: the intersection of these cones results in the visual hull
Fig. 8
Fig. 8
An example image from the HumanEva dataset used to validate markerless systems within computer vision. White dots indicate the location of tracked reflective markers and the cyan lines represent the defined skeleton model fit to the marker data. Although useful as an early benchmark for markerless tracking systems, the dataset has clear limitations for assessing the quality of any markerless tracking results, especially in the context of biomechanics. Notice that the markers are attached to clothing, marker clusters are not utilised, and the joint centres inferred from the fitted skeleton are not closely aligned with how the person appears in the image (e.g. right elbow and hip joints). See further information in the text
See this image and copyright information in PMC

References

    1. Muybridge E. Complete human and animal locomotion (all 781 plates from the 1887 animal locomotion) In: Cappozzo A, Marchetti M, Tosi V, editors. Biolocomotion: a century of research using moving pictures. Rome: Promograph. 1979. p. 69.
    1. Cimolin V, Galli M. Summary measures for clinical gait analysis: a literature review. Gait Posture. 2014;39(4):1005–1010. doi: 10.1016/j.gaitpost.2014.02.001. - DOI - PubMed
    1. Cosker D, Eisert P, Helzle V. Facial capture and animation in visual effects. In: Magnor MA, Grau O, Sorkine-Hornung O, Theobalt C, editors. Digital representations of the real world: how to capture, model, and render visual reality. Boca Raton, FL: CRC Press; 2015. pp. 311–321.
    1. Bezodis NE, Salo AIT, Trewartha G. Relationships between lower-limb kinematics and block phase performance in a cross section of sprinters. Eur J Sport Sci. 2015;15(2):118–124. doi: 10.1080/17461391.2014.928915. - DOI - PubMed
    1. Ford KR, Myer GD, Toms HE, Hewett TE. Gender differences in the kinematics of unanticipated cutting in young athletes. Med Sci Sports Exerc. 2005;37(1):124–129. doi: 10.1249/01.MSS.0000150087.95953.C3. - DOI - PubMed