Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain

Abstract

The human body is a template for many state-of-the-art prosthetic devices and sensors. Perceptions of touch and pain are fundamental components of our daily lives that convey valuable information about our environment while also providing an element of protection from damage to our bodies. Advances in prosthesis designs and control mechanisms can aid an amputee's ability to regain lost function but often lack meaningful tactile feedback or perception. Through transcutaneous electrical nerve stimulation (TENS) with an amputee, we discovered and quantified stimulation parameters to elicit innocuous (non-painful) and noxious (painful) tactile perceptions in the phantom hand. Electroencephalography (EEG) activity in somatosensory regions confirms phantom hand activation during stimulation. We invented a multilayered electronic dermis (e-dermis) with properties based on the behavior of mechanoreceptors and nociceptors to provide neuromorphic tactile information to an amputee. Our biologically inspired e-dermis enables a prosthesis and its user to perceive a continuous spectrum from innocuous to noxious touch through a neuromorphic interface that produces receptor-like spiking neural activity. In a Pain Detection Task (PDT), we show the ability of the prosthesis and amputee to differentiate non-painful or painful tactile stimuli using sensory feedback and a pain reflex feedback control system. In this work, an amputee can use perceptions of touch and pain to discriminate object curvature, including sharpness. This work demonstrates possibilities for creating a more natural sensation spanning a range of tactile stimuli for prosthetic hands.

Fig. 1.. Prosthesis system diagram.

Tactile information from object grasping is transformed into a neuromorphic signal through the prosthesis controller. The neuromorphic signal is used to transcutaneously stimulate peripheral nerves of an amputee to elicit sensory perceptions of touch and pain.

Fig. 2.. Multilayered e-dermis design and characterization.

(A) The multilayered e-dermis is made up of conductive and piezoresistive textiles encased in rubber. A dermal layer of two piezoresistive sensing elements are separated from the epidermal layer, which has one piezoresistive sensing element, with a 1 mm layer of silicone rubber. The e-dermis was fabricated to fit over the fingertips of a prosthetic hand. (B) The natural layering of mechanorecptors in healthy glabrous skin makes use of both rapidly (RA) and slowly adapting (SA) receptors to encode the complex properties of touch. Also present in the skin are free nerve endings (nociceptors) that are primarily responsible for conveying the sensation of pain in the fingertips. (C) The prosthesis with e-dermis fingertip sensors grasps an object. (D) The epidermal layer of the multilayered e-dermis design is more sensitive and has a larger change in resistance compared to the dermal layer. (E) Differences in sensing layer outputs are captured during object grasping and can be used for adding dimensionality to the tactile signal.

Fig. 3.. Sensory feedback and perception.

(A) Median and ulnar nerve sites on the amputee’s residual limb and the corresponding regions of activation in the phantom hand due to TENS. (B) Psychophysical experiments quantify the perception of the nerve stimulation including detection and (C) discrete frequency discrimination thresholds. In both cases the stimulation amplitude was held at 1.4 mA. (D) The perception of the nerve stimulation was largely a tactile pressure on the activated sites of the phantom hand although sensations of electrical tingling also occured. (E) The quantification of pain from nerve stimulation shows that the most noxious sensation is perceived at higher stimulation pulse widths with frequencies in the 10 – 20 Hz range. (F) Contralateral somatosensory cortex activation during nerve stimulation shows relevant cortical representation of sensory perception in the amputee participant (movie S1).

Fig. 4.. E-dermis and neuromorphic tactile response from different objects.

(A) Three different objects, with equal width but varying curvature, are used to elicit tactile responses from the multilayered e-dermis. (B) The pressure heatmap from the fingertip sensor on a prosthetic hand during grasping of each object and (C) the corresponding pressure profile for each of the sensing layers. (D) The pressure profiles are converted to the input current, I, for the Izhikevich neuron model for sensory feedback to the amputee user (movie S2). Note the highly localized pressure during the grasping of Object 3 and the resulting nociceptor neuromorphic stimulation pattern, which is realized through changes in both stimulation pulse width and the neuromorphic model parameters.

Fig. 5.. Prosthesis grasping and control.

To demonstrate the ability of the prosthesis to determine safe (innocuous) or unsafe (painful) objects, we performed the pain detection task (PDT). The objects are (A) Object 1, (B) Object 2, and (C) Object 3, each of which are defined by their curvature. In the case of a painful object (Object 3), the prosthesis detects the sharp pressure and releases its grip through its pain reflex (movie S3).

Fig. 6.. Tactile features for prosthesis perception.

To determine which object is being touched during grasping, we implemented LDA to discriminate between the independent classes. As input features into the algorithm, we used (A) sensor pressure values, (B) the rate of change of the pressure signal, and (C) the number of active sensing elements during loading.

Fig. 7.. Real-time prosthesis pain perception.

(A) The LDA classifier’s accuracy across the various conditions and (B) the percentage of trials where the prosthesis perceived pain during the online PDT. Note the high percentage of detected pain during the PDT for Object 3. (C) Pain reflex time of the prosthesis, using the rate of change of the pressure signal to determine object contact and release, compared to previously published data of pain reflex time in healthy adults (28).

Fig. 8.. Innocuous (mechanoreception) and noxious (nociception) prosthesis sensing and discrimination in an amputee.

(A) The amputee could discriminate which region of his phantom hand was activated, if at all. (B) Perception of pain increases with decreasing radius of curvature (i.e. increase in sharpness) for the objects presented to the prosthetic hand. (C) Discrimination accuracy shows the participant’s ability to reliably identify each object presented to the prosthesis based purely on the sensory feedback from the neuromorphic stimulation. (D) Results from the PDT during user controlled movements, with pain reflex enabled.