Real-time decoding of brain responses to visuospatial attention using 7T fMRI

Abstract

Brain-Computer interface technologies mean to create new communication channels between our mind and our environment, independent of the motor system, by detecting and classifying self regulation of local brain activity. BCIs can provide patients with severe paralysis a means to communicate and to live more independent lives. There has been a growing interest in using invasive recordings for BCI to improve the signal quality. This also potentially gives access to new control strategies previously inaccessible by non-invasive methods. However, before surgery, the best implantation site needs to be determined. The blood-oxygen-level dependent signal changes measured with fMRI have been shown to agree well spatially with those found with invasive electrodes, and are the best option for pre-surgical localization. We show, using real-time fMRI at 7T, that eye movement-independent visuospatial attention can be used as a reliable control strategy for BCIs. At this field strength even subtle signal changes can be detected in single trials thanks to the high contrast-to-noise ratio. A group of healthy subjects were instructed to move their attention between three (two peripheral and one central) spatial target regions while keeping their gaze fixated at the center. The activated regions were first located and thereafter the subjects were given real-time feedback based on the activity in these regions. All subjects managed to regulate local brain areas without training, which suggests that visuospatial attention is a promising new target for intracranial BCI. ECoG data recorded from one epilepsy patient showed that local changes in gamma-power can be used to separate the three classes.

Figure 1. Timeline of the experiment.

The localizer and feedback data are acquired in the same fMRI run.

Figure 2. The visual stimuli.

Figure 3. The control signals (CS) for all subjects.

(Subject 1-9 from left to right and top to bottom.) Light and dark gray represent right-sided and left-sided attention respectively. The blocks have been shifted 3 TRs (4.9s) to compensate for the hemodynamic delay. The last plot shows the average control signal over all subjects, with the standard deviation shown in white.

Figure 4. Average control signal during the central, right-sided attention and left-sided attention trials.

The averages are shown both for the actual control signal during feedback and the control signal computed offline using the localizing data. The standard deviation is shown in gray.

Figure 5. ROC curves plotted for the control signal classification of Subject 2 over varying thresholds.

Figure 6. Number of subjects having a specific image volume correctly classified.

Each row represents one of the 10 trials, and each column a time point (not adjusted for hemodynamic delay) in that trial. The curves show, for all time points, how many trials would be correctly classified if based only on this particular volume.

Figure 7. The group activation pattern.

Red represents t-values from the contrast ‘attend right-attend left’ while blue represents ’attend left-attend right’.

Figure 8. The individual subjects'activation patterns.

The patterns both during the localizer part and the feedback part are displayed on the MNI brain. The red and blue color scales represent t-values from the contrasts ’attend right minus attend left’ and ’attend left minus attend right’.

Figure 9. Activations and deactivations.

The group t-values higher than 2.5 (see Figure 7) are separated into areas showing activation versus areas showing deactivation relative to the central attention task. The upper half shows the contrast ’attend right-attend left’ and the lower half ’attend left-attend right’. Red represents voxels whose contribution comes from increased activity, blue the voxels showing deactivation during attention to the other side, and green voxels showing both these effects.

Figure 10. ECoG electrode selections.

(a) The fMRI group activation pattern (t , red: ’right-left’, blue: ’left-right’). (b) The yellow markers show the electrodes' locations on the cortical surface. On top of the markers it is shown in how many of the leave-one-out tests the electrode was included. Red, blue and green represent right, left and center attention.