Towards neural co-processors for the brain: combining decoding and encoding in brain-computer interfaces

Abstract

The field of brain-computer interfaces is poised to advance from the traditional goal of controlling prosthetic devices using brain signals to combining neural decoding and encoding within a single neuroprosthetic device. Such a device acts as a 'co-processor' for the brain, with applications ranging from inducing Hebbian plasticity for rehabilitation after brain injury to reanimating paralyzed limbs and enhancing memory. We review recent progress in simultaneous decoding and encoding for closed-loop control and plasticity induction. To address the challenge of multi-channel decoding and encoding, we introduce a unifying framework for developing brain co-processors based on artificial neural networks and deep learning. These 'neural co-processors' can be used to jointly optimize cost functions with the nervous system to achieve desired behaviors ranging from targeted neuro-rehabilitation to augmentation of brain function.

Figure 1:. Neural Co-Processor for the Brain for Restoring and Augmenting Function.

A deep recurrent artificial network is used to map input neural activity patterns in one set of regions to output stimulation patterns in other regions (“Co-Processor Network” or CPN). The CPN’s weights are optimized to minimize brain-activity-based error (between stimulation patterns and target neural activity patterns when known), or more generally, to minimize behavioral error or task error using another network, an emulator network. The emulator network is also a deep recurrent network that is pre-trained through backpropagation to learn the biological transformation from stimulation or neural activity patterns at the stimulation site to the resulting output behaviors. During CPN training, errors are backpropagated through the emulator network to the CPN to adapt the CPN’s weights but not the emulator network’s weights. The trained CPN thus produces optimal stimulation patterns that minimize behavioral error, thereby creating a goal-directed artificial information processing pathway between the input and output regions. The CPN also promotes neuroplasticity between weakly connected regions, leading to neural augmentation or targeted rehabilitation. External information from artificial sensors or other information sources can be integrated into the CPN’s information processing as additional inputs to the neural network and outputs can be computed for external actuators as well. The example here shows the CPN creating a new information processing pathway between prefrontal cortex and motor cortex, bypassing an intermediate area affected by brain injury or stroke. The CPN is trained to transform movement intentions in the prefrontal cortex to appropriate movement-related stimulation patterns in the motor cortex for restoration of movement and rehabilitation.