Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Dec 16;21(24):8409.
doi: 10.3390/s21248409.

Fusion Models for Generalized Classification of Multi-Axial Human Movement: Validation in Sport Performance

Rajesh Amerineni  1 , Lalit Gupta  1 , Nathan Steadman  2 , Keshwyn Annauth  2 , Charles Burr  2   3 , Samuel Wilson  2   4 , Payam Barnaghi  5   6 , Ravi Vaidyanathan  2   3   4   6
Affiliations

Fusion Models for Generalized Classification of Multi-Axial Human Movement: Validation in Sport Performance

Rajesh Amerineni et al. Sensors (Basel). .

Abstract

We introduce a set of input models for fusing information from ensembles of wearable sensors supporting human performance and telemedicine. Veracity is demonstrated in action classification related to sport, specifically strikes in boxing and taekwondo. Four input models, formulated to be compatible with a broad range of classifiers, are introduced and two diverse classifiers, dynamic time warping (DTW) and convolutional neural networks (CNNs) are implemented in conjunction with the input models. Seven classification models fusing information at the input-level, output-level, and a combination of both are formulated. Action classification for 18 boxing punches and 24 taekwondo kicks demonstrate our fusion classifiers outperform the best DTW and CNN uni-axial classifiers. Furthermore, although DTW is ostensibly an ideal choice for human movements experiencing non-linear variations, our results demonstrate deep learning fusion classifiers outperform DTW. This is a novel finding given that CNNs are normally designed for multi-dimensional data and do not specifically compensate for non-linear variations within signal classes. The generalized formulation enables subject-specific movement classification in a feature-blind fashion with trivial computational expense for trained CNNs. A commercial boxing system, 'Corner', has been produced for real-world mass-market use based on this investigation providing a basis for future telemedicine translation.

Keywords: CNNs; DTW; IMUs; deep learning; human performance; motion tracking; sensor fusion; sports biomechanics; wearable sensors.

PubMed Disclaimer

Conflict of interest statement

C.B. and R.V. are co-founders of the company Corner, which has released the commercial boxing system described in Section 9.

Figures

Figure 1
Figure 1
The Vector Input (VI) model.
Figure 2
Figure 2
The Local Matrix Input (LMI) model.
Figure 3
Figure 3
The Global Matrix Input (GMI) model.
Figure 4
Figure 4
The Global Cuboid Input (GCI) model.
Figure 5
Figure 5
The DTW-1 classification model.
Figure 6
Figure 6
The DTW-2 classification model.
Figure 7
Figure 7
The DTW-3 classification model.
Figure 8
Figure 8
Block diagram of a CNN with two convolution layers and a pooling layer.
Figure 9
Figure 9
The CNN-1 classification model.
Figure 10
Figure 10
The CNN-2 classification model.
Figure 11
Figure 11
The CNN-3 classification model.
Figure 12
Figure 12
The CNN-4 classification model.
Figure 13
Figure 13
Data collection example: left jab punch (pad) (sensors under glove on wrist).
Figure 14
Figure 14
Data collection example: front right kick (sensors strapped to each ankle).
Figure 15
Figure 15
Superimposed strikes of the boxing Right Hand Shadow Hook ensemble acquired from the x-axis of the accelerometer.
Figure 16
Figure 16
Superimposed strikes of the taekwondo Right Axe Contact Kick ensemble acquired from the x-axis of the accelerometer.
Figure 17
Figure 17
Training parameterization time for DTW and CNN models.
Figure 18
Figure 18
Corner Boxing System: Sensor with dimensions (top), sensor, phone app and wrist attachment (middle), and sample group training session with data from several users compiled and displayed simultaneously (bottom).
See this image and copyright information in PMC

References

    1. Burridge J.H., Lee A.C.W., Turk R., Stokes M., Whitall J., Vaidyanathan R., Clatworthy P., Hughes A.-M., Meagher C., Franco E., et al. Telehealth, Wearable Sensors, and the Internet: Will They Improve Stroke Outcomes Through Increased Intensity of Therapy, Motivation, and Adherence to Rehabilitation Programs? J. Neurol. Phys. Ther. 2017;41((Suppl. 3)):S32–S38. doi: 10.1097/NPT.0000000000000183. - DOI - PubMed
    1. Woodward R.B., Shefelbine S.J., Vaidyanathan R. Pervasive Monitoring of Motion and Muscle Activation: Inertial and Mechanomyography Fusion. IEEE/ASME Trans. Mechatron. 2017;22:2022–2033. doi: 10.1109/TMECH.2017.2715163. - DOI
    1. Ahmadi A., Mitchell E., Richter C., Destelle F., Gowing M., O’Connor N.E., Moran K. Toward Automatic Activity Classification and Movement Assessment during a Sports Training Session. IEEE Internet Things J. 2015;2:23–32. doi: 10.1109/JIOT.2014.2377238. - DOI
    1. Camomilla V., Bergamini E., Fantozzi S., Vannozzi G. Trends Supporting the In-Field Use of Wearable Inertial Sensors for Sport Performance Evaluation: A Systematic Review. Sensors. 2018;18:873. doi: 10.3390/s18030873. - DOI - PMC - PubMed
    1. Bianchi V., Bassoli M., Lombardo G., Fornacciari P., Mordonini M., De Munari I. IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment. IEEE Internet Things J. 2019;6:8553–8562. doi: 10.1109/JIOT.2019.2920283. - DOI

LinkOut - more resources