Wearable Nail Deformation Sensing for Behavioral and Biomechanical Monitoring and Human-Computer Interaction

Abstract

The dynamics of the human fingertip enable haptic sensing and the ability to manipulate objects in the environment. Here we describe a wearable strain sensor, associated electronics, and software to detect and interpret the kinematics of deformation in human fingernails. Differential forces exerted by fingertip pulp, rugged connections to the musculoskeletal system and physical contact with the free edge of the nail plate itself cause fingernail deformation. We quantify nail warpage on the order of microns in the longitudinal and lateral axes with a set of strain gauges attached to the nail. The wearable device transmits raw deformation data to an off-finger device for interpretation. Simple motions, gestures, finger-writing, grip strength, and activation time, as well as more complex idioms consisting of multiple grips, are identified and quantified. We demonstrate the use of this technology as a human-computer interface, clinical feature generator, and means to characterize workplace tasks.

Figure 1

Anatomy of fingernail. (A) Scanning electron micrograph of a fingernail cross-section. (B) Index finger photograph and X-ray of bone structures with connection schematic.

Figure 2

Fingertip surface profile measurements. DIC images of fingernail during normal force test. (A) Left to Right: Z-axis displacement, strain along X (center), and strain along Y (right) directions. (B) Z-axis displacement and strain profile at five newtons with and without a strain gauge. (C) Z-axis displacement during shear forces.

Figure 3

Schematic diagram of the wearable system. (A) Electronics with silicone prosthetic on the nail. (B) Electronics on the skin with for debugging. Schematics of the fingertip model in panel A have been reproduced with permission by Bucknell Webb.

Figure 4

Examples of strain response data. (A) Strain response data for a series of flat normal force presses against the surface of a dynamometer (left) and strain vs. force curves for these same actions (right). (B) Strain response for a series of normal force, forty-five-degree angle finger presses on a glass surface with the finger pulp first distally and then proximally displaced (see Supplementary Movie 1). (C) Responses when the finger is pushed away and the onychodermal band pulls down on the distal end of the nail imparting bending (tensile) stress. (D) Strain responses for a series of sliding motions toward from the subject under a normal force on a glass surface producing compressive stress. (E) Strain responses to a series of left and right lateral movements under a normal force on the glass surface. (F) A set of three finger extensions followed by three flexion movements.

Figure 5

Examples of strain responses during precision and power grips. (A) Strain and force responses during a set of precision pinch grips. (B) Strain versus force responses during precision pinch. (C) Responses for a power grip session. (D) Strain versus force responses during power grip.

Figure 6

Detection of finger movement idioms. (A) T-SNE visualization showing separation of strain signals color coded according to their true values. (B) Confusion matrix of correctly and incorrectly predicted finger digits for the test set. The absolute number of trials is reported.

Figure 7

Detection of hand activities. (A) Time series of probability activations corresponding to five actions: using a screwdriver, turning a doorknob, turning a key, screwing and unscrewing a nut, and rest. (B) Confusion matrix of correctly and incorrectly predicted hand activities for the test set. The absolute number of trials is reported.