A Multi-Modal Under-Sensorized Wearable System for Optimal Kinematic and Muscular Tracking of Human Upper Limb Motion

Abstract

Wearable sensing solutions have emerged as a promising paradigm for monitoring human musculoskeletal state in an unobtrusive way. To increase the deployability of these systems, considerations related to cost reduction and enhanced form factor and wearability tend to discourage the number of sensors in use. In our previous work, we provided a theoretical solution to the problem of jointly reconstructing the entire muscular-kinematic state of the upper limb, when only a limited amount of optimally retrieved sensory data are available. However, the effective implementation of these methods in a physical, under-sensorized wearable has never been attempted before. In this work, we propose to bridge this gap by presenting an under-sensorized system based on inertial measurement units (IMUs) and surface electromyography (sEMG) electrodes for the reconstruction of the upper limb musculoskeletal state, focusing on the minimization of the sensors' number. We found that, relying on two IMUs only and eight sEMG sensors, we can conjointly reconstruct all 17 degrees of freedom (five joints, twelve muscles) of the upper limb musculoskeletal state, yielding a median normalized RMS error of 8.5% on the non-measured joints and 2.5% on the non-measured muscles.

Keywords: IMUs; Sensor Fusion; human multimodal motion tracking; optimal design; sEMG sensors; upper limb; wearable sensing.

Figure 1

Schematic flow of the estimation procedure. First temporal signals are mapped on the weight vector through the fPC bases (Encoding). After that, Minimum Variance Estimation (MVE) fuses the encoded measures with a priori knowledge to estimate for the missing part of measures. In the end, the estimated weight vector is converted back to the temporal domain (Decoding).

Figure 2

EMG sensor placement in accordance with SENIAM recommendations (back and front views of the right arm). In blue, the muscles used as measures in the MVE algorithm; in red, the estimated muscles.

Figure 3

Kinematic model of the human arm (the angle $q_3$ is directed outwards).

Figure 4

Different views of the complete sensor setup (including the ground truth sensors) used during the experimental phase. The full-body view of the system — composed by the Delsys Bagnoli EMG system (Delsys Inc., Natick, MA, USA), the Xsens MTw Awinda (Movella Inc., Henderson, NV, USA) wearable system and the two LSM9DS1 inertial sensors embedded in Arduino Nano 33 BLE boards (Arduino S.r.l., Monza, Italy) — is shown in (a,b). A detail of the IMUs positioning is depicted in (c).

Figure 5

Normalized RMS Error computed for each DoF (measured DoFs in blue, non-measured DoFs in red).

Figure 6

Example of MVE on a movement of the test dataset (in blue: reference movement; in green: movement reconstruction with fPCs; in red: movement obtained through MVE); * = non−measured DoFs.