Restoring sensorimotor function through intracortical interfaces: progress and looming challenges

Abstract

The loss of a limb or paralysis resulting from spinal cord injury has devastating consequences on quality of life. One approach to restoring lost sensory and motor abilities in amputees and patients with tetraplegia is to supply them with implants that provide a direct interface with the CNS. Such brain-machine interfaces might enable a patient to exert voluntary control over a prosthetic or robotic limb or over the electrically induced contractions of paralysed muscles. A parallel interface could convey sensory information about the consequences of these movements back to the patient. Recent developments in the algorithms that decode motor intention from neuronal activity and in approaches to convey sensory feedback by electrically stimulating neurons, using biomimetic and adaptation-based approaches, have shown the promise of invasive interfaces with sensorimotor cortices, although substantial challenges remain.

Figure 1 |. Idealized bidirectional brain–machine interface for prosthetic control.

Neural signals from motor-related areas of the brain — for example, the primary motor cortex (M1) — that encode the intended movement (motor intent) are decoded and used to control the movement of the prosthetic limb. Sensors on the prosthetic limb convey information about movements of the limb and any objects with which it comes into contact. The output of these sensors is converted into patterns of electrical stimulation (stimulus pulses), which are delivered to sensory areas of the brain — for example, the primary somatosensory cortex (S1) — via chronically implanted arrays of electrodes.

Figure 2 |. Seven-DOF control of a prosthetic limb for reaching and grasping.

a | A patient with tetraplegia had two electrode arrays (Blackrock Microsystems, Salt Lake City, Utah, USA) implanted into the primary motor cortex. b | The patient controlled the position and orientation of the modular prosthetic limb in order to grasp objects. c | The patient’s performance on grasping tasks improved over time, even as the task complexity increased and the degree of computer assistance decreased. The grey trace shows chance level given the level of assistance. d–f | State-of-the-art anthropomorphic prosthetic hands. The modular prosthetic limb (part d) developed by DARPA (Defense Advanced Research Projects Agency, USA) and the Johns Hopkins University Applied Physics Laboratory (Laurel, Maryland, USA) has 19 degrees of freedom (DOFs) in the hand alone (one less than in the actual hand) and over 100 sensors in the arm and hand. The DLR Hand II (part e) from the Institute of Robotics and Mechatronics of the German Aerospace Center (Wessling, Germany) has 15 active DOFs and a wide range of position, contact and torque sensors. The i-limb ultra revolution (part f) from Touch Bionics (Mansfield, Massachusetts, USA) features a powered rotating thumb that moves automatically into position. All five digits move independently, bending at the joints to fit around the shape of whatever object is grasped. It can also be automatically configured into any of 24 grip patterns. Images in parts a and b courtesy of University of Pittsburgh Medical Center, USA. Part c is based on data from REF. . Image in part d courtesy of the Johns Hopkins University Applied Physics Laboratory, USA. Image in part e courtesy of DLR, Institute of Robotics and Mechatronics, Germany. Image in part f courtesy of Advanced Arm Dynamics, USA.

Figure 3 |. Offline electromyography-based predictions during flexion and extension isometric wrist torque.

Most brain–machine interfaces (BMIs) use signals from the primary motor cortex (M1) to make predictions about limb kinematics, typically the position or the velocity of the hand. Force- and muscle-related information is also richly represented in M1 and offers a different approach to movement control with a BMI. In this example, a monkey was trained to place its hand in an isometric force transducer and to exert wrist force in either the flexion (flx.) or extension (ext.) direction. The monkey learned to control a cursor, the movement of which was determined by the forces exerted on the transducer, to place it in a series of targets displayed on a video screen. The resulting force signal is shown by the white trace. At the same time, recordings were made from 77 single- and multi-unit neural recordings in the hand area of M1. The firing rate of each unit, normalized to its maximum rate within this period, is indicated by a colour code, with each line corresponding to a different unit. Note the bursts of unit activity that slightly precede each torque pulse. Electromyography (EMG) signals from the four wrist muscles (red traces) were recorded simultaneously and correlated well with the exerted force. The investigators computed the decoder using data collected in this task and used these data to make EMG predictions for each of the four wrist muscles (blue traces). Real-time predictions of muscle activity have been used to control electrical stimulation of muscles aimed at restoring voluntary arm movement in patients with tetraplegia^,. Similar predictions might also be used to control the movement of a robotic arm by incorporating an appropriate musculoskeletal model. Such an approach would enable one to control the impedance of the limb during movement as well as the motion itself. The figure is reproduced, with permission, from REF. © (2010) Academic Press, Elsevier.

Figure 4 |. Conveying information about contact location and pressure using a biomimetic approach.

a | Monkeys were mechanically touched on the hand twice in a row and were trained to judge whether the second indentation was to the left or right (lateral or medial) of the first one (left panel). (Each blue dot corresponds to the location of a mechanical indentation.) On a subset of trials (hybrid trials), one of the indentations was replaced with an intracortical microstimulation (ICMS) pulse train delivered to an electrode with a receptive field centred at the site of the indentation (red shaded area). Mean performance on mechanical and hybrid trials is shown in the right panel. Although animals performed better on mechanical trials than on hybrid trials, performance in the latter was considerably (and significantly) above chance (0.5). In other words, ICMS through individual electrodes produces percepts that are localized to a patch of skin corresponding to their receptive field. b | Monkeys were trained to detect indentations of the skin or ICMS pulse trains delivered to the primary somatosensory cortex (S1); they were also trained to discriminate the intensity of these mechanical or electrical stimuli using the same behavioural paradigm. The graph on the left shows the mapping between pressure magnitude (that is, the depth of the skin indentation) and the ICMS pulse amplitude that yielded percepts of equal sensory magnitude (for example, that were equally detectable or equally discriminable from an equivalently detectable standard stimulus). The graph on the right shows that animals performed equally well on a force discrimination task when the tactile stimuli were delivered to the actual finger (blue dots and trace) as when they were delivered to a prosthetic finger (red dots and trace, with each trace corresponding to a different electrode). Data in right panel of part a from REF. . Part b is modified from REF. .

Figure 5 |. Bidirectional interface based on learned associations.

The monkey explored virtual space with a virtual hand (or avatar) to find which of three visually identical targets had the right ‘texture’. The avatar was controlled by signals recorded in the primary motor cortex (M1); tactile sensations of texture were elicited by stimulating the primary somatosensory cortex (S1) with patterned pulse trains. When the avatar hovered over two of the three visual targets, the monkey received one of two pulse trains of intracortical microstimulation (ICMS). The third target did not trigger ICMS. The animal was successful if it correctly identified the target associated with the rewarded ICMS train. Over time, the animal learned to identify the rewarded target. This study demonstrates that animals can learn to discriminate between different patterns of sensory stimulation applied to S1 and combined, for the first time, motor decoding and ICMS feedback. The figure is modified, with permission, from REF. © (2011) Macmillan Publishers Ltd. All rights reserved.