Invasive neurophysiology and whole brain connectomics for neural decoding in patients with brain implants

Abstract

Brain computer interfaces (BCI) provide unprecedented spatiotemporal precision that will enable significant expansion in how numerous brain disorders are treated. Decoding dynamic patient states from brain signals with machine learning is required to leverage this precision, but a standardized framework for identifying and advancing novel clinical BCI approaches does not exist. Here, we developed a platform that integrates brain signal decoding with connectomics and demonstrate its utility across 123 hours of invasively recorded brain data from 73 neurosurgical patients treated for movement disorders, depression and epilepsy. First, we introduce connectomics-informed movement decoders that generalize across cohorts with Parkinson's disease and epilepsy from the US, Europe and China. Next, we reveal network targets for emotion decoding in left prefrontal and cingulate circuits in DBS patients with major depression. Finally, we showcase opportunities to improve seizure detection in responsive neurostimulation for epilepsy. Our platform provides rapid, high-accuracy decoding for precision medicine approaches that can dynamically adapt neuromodulation therapies in response to the individual needs of patients.

Figure 1. Use of brain signal decoding and adaptation of neurostimulation implemented in the py_neuromodulation platform.

Neural data streams (a) can either be simulated from stored offline storage or streamed in real-time through direct connection to neural implants. Preprocessing (b) includes re-referencing, notch-filtering, downsampling, normalization, artifact detection and more, and was optimized for causal and computational efficient application. Multiple brain signal feature modalities (c) can be extracted that are relevant for invasive decoding: oscillatory activity, temporal waveform shape, oscillatory bursts, nonlinear dynamics, periodic and aperiodic power spectral components and more (Supplementary Table 1). Features can be mapped in space (d) for patient individual or across-patient decoding and consecutive adjustment of therapeutic delivery. Cross-validation, model evaluation metrics, and model architectures can be specified through the scikit-learn or alternative machine learning frameworks.

Figure 2. Movement decoding across patients, cohorts, diseases, movement types, and stimulation conditions.

(a)Data from four cohorts with different diseases and movement types were used for decoding (1480 channels, 56 patients). (b) Individual recording locations with color-coded movement decoding classification performances. (c) Performances from patient individual 3-fold cross-validation. (d) Movement detection rates are defined as 300 ms consecutively correct classification during movement (98 ± 4 % for best channels across all patients). (e) In PD, mean channel performances negatively correlated with motor sign severity (UPDRS-III). (f) Exemplar time-series with DBS on ECoG raw data in a representative subject from the Berlin cohort. (g) Sample-wise performances OFF and ON clinically effective subthalamic 130 Hz DBS in six PD patients from Berlin (all above chance). To demonstrate the utility of py_neuromodulation for across-patient decoding, three alternative pipelines integrate channel selection and neural signals: (h) Spatial interpolation to a common grid in MNI space; (i) channel selection based on normative fMRI connectivity correlation to a predefined optimal decoding network; (j) embedding (exemplar subject shown) using contrastive learning with a convolutional neural network using CEBRA. (k) shows embedding consistency from each to every other patient via linear identifiability. All three methods achieved high decoding accuracies within and across cohorts for sample-wise balanced accuracy (l) and movement detection rates (m). Patient to patient training (n) revealed interindividual variability for training other subjects vs. being trained on other subjects (n is ordered according to performance, subjects Berlin-002 and Beijing-005 were the best trainers). To demonstrate the ability to decode movements without patient individual training, we prospectively recruited one subject in Berlin and decoded movements using pretrained models based on all previous subjects of the Berlin cohort (o). Real-time decoding performances (p) show that py_neuromodulation can facilitate prospective real-time decoding based on various training cohorts without requiring individual training with high above chance sample-wise balanced accuracy and movement detection rates.

Figure 3. Emotion decoding using LFP signals from subgenual cingulate cortex in patients with treatment-resistant depression.

Eight subjects undergoing DBS surgery performed an emotion task (a) with visual stimuli of negative, neutral, and positive valence. Electrode locations are visualized alongside the anterior- (red and light blue) and subcallosal cingulate cortex (white and dark blue, Harvard-Oxford atlas). Balanced decoding accuracies (c) rose from 150 ms after onset, peaked at 600 ms and decayed until 1600 ms post stimulus. Best channel performances showed above chance emotion decoding across subjects (all p<0.05). (e) Best performance channels revealed highest feature importances for FFT gamma features followed by different temporal waveform shape features. (f) Best performances correlated with DBS induced Beck Depression Inventory (BDI) changes 24 months following DBS implantation (rho=0.79, p=0.01). Performances were, however, not significantly correlated to baseline BDI scores. (g) Significant fiber-tracks, FDR (False Discovery Rate) corrected with , predicting emotional state decoding performances showed a clear relation to the left prefrontal cortex. This is reflected in functional and structural connectivity for all patient channels (g) and particularly visible for fiber filtering (h) and fMRI maps (i). The estimated best therapeutic stimulation target from Fox et al 2014 is additionally displayed. All three connectivity models (fMRI, dMRI, fiber filtering) could cross-predict left out channel decoding performances (Supplementary Fig. 1,2).

Figure 4. Predictive parameter identification for seizure detection in responsive neurostimulation (RNS).

(a) RNS data was either acquired from two lead depth electrodes (eight patients) or a cortical electrode in addition to a depth electrode (one patient). Two lead depth electrodes are visualized using Lead-DBS for an exemplar patient. The Harvard-Oxford atlas parcellation shows the lingual gyrus (white) and the left lateral occipital cortex superior division (red) that are penetrated by the electrodes. (b) In an exemplary recording, from a baseline programming epoch without stimulation artifacts, ictal onset characteristics across the four recording channels are displayed (top). RNS detectors can produce false positive seizure detections even in the absence of seizure activity (bottom) that can occur so frequently that no further therapy is provided (maximum stimulation number reached), even though no ictal activity is present (as defined by the epileptologist annotation). (c) FFT and line-length features were computed similar to the embedded RNS algorithm with py_neuromodulation. The exemplary ictal recording shows clear seizure induced changes. (d) By combining these features with expert seizure annotations, optimal detection parameters were extracted in a grid-search to optimize the F1 score for seizure detection. (e) The RNS patient data management system (PDMS) provides the “SimpleStart” algorithm, in which detection programming settings are automatically inferred based on a single ictal event. Instead, we propose using optimized parameters based on machine learning models that are trained on expert annotations across hundreds of events with py_neuromodulation. This may hold the potential for improved true negative predictions and increased F1 scores (f). It can become even more promising when more complex feature sets and machine learning algorithms will be implemented in the brain implant. (g) We show that the extension to multiple additional features (e.g. bursting, fooof, temporal waveform shape, etc.) can further increase performance with robust three-fold cross-validation. (h) Gradient-boosted decision trees using the XGBOOST framework were outperforming linear models and support vector machines for seizure prediction using all computed features and may provide an optimal trade-off between complexity and interpretability in future neurotechnology.