Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery

Abstract

Motor execution, observation, and imagery are important skills used in motor learning and rehabilitation. The neural mechanisms underlying these cognitive-motor processes are still poorly understood. We used a simultaneous recording of functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) to elucidate the differences in neural activity across three conditions requiring these processes. Additionally, we used a new method called structured sparse multiset Canonical Correlation Analysis (ssmCCA) to fuse the fNIRS and EEG data and determine the brain regions of neural activity consistently detected by both modalities. Unimodal analyses revealed differentiated activation between conditions; however, the activated regions did not fully overlap across the two modalities (fNIRS: left angular gyrus, right supramarginal gyrus, as well as right superior and inferior parietal lobes; EEG: bilateral central, right frontal, and parietal). These discrepancies might be because fNIRS and EEG detect different signals. Using fused fNIRS-EEG data, we consistently found activation over the left inferior parietal lobe, superior marginal gyrus, and post-central gyrus during all three conditions, suggesting that our multimodal approach identifies a shared neural region associated with the Action Observation Network (AON). This study highlights the strengths of using the multimodal fNIRS-EEG fusion technique for studying AON. Neural researchers should consider using the multimodal approach to validate their findings.

Figure 1

Experimental setup for each condition: (A) Execution, (B) observation, (C) imagination. Written permission has been taken for publication of participant pictures.

Figure 2

fNIRS probe placements and the ROI assignments. Written permission for publication of participant pictures has been taken.

Figure 3

Unimodal fNIRS results. This is a visual representation of the HbO and HbR Z-scores during Execution (ME), Observation (MO), and Imagination (MI). HbO values on Y-axis range from − 1.5 indicated by cooler color (blue) to 0.1 indicated by warmer color (red). Highlighted regions indicate an increase in HbO and a decrease in HbR Z-scores. During ME, positive Z-scores for HbO and negative Z-scores for HbR were found in left PRCG, SPL, AG, and right POCG, SPL, SMG (A,E). During MO, positive HbO and negative HbR were found over Left PRCG, POCG, and right POCG, SPL (B,F). During MI, positive HbO and negative HbR were found in left and right POCG (C,G).

Figure 4

Unimodal EEG results. This is a visual representation for power synchronization. Synchronization is indicated by warmer colors (red), while desynchronization is indicated by cooler colors (blue). Significantly greater left AG activation were found during ME, compared to the MO and MI conditions (ps < 0.05); Greater right SMG activation was found during MO compared to ME and MI (ps < 0.05), and greater left POCG were found during MO compared to ME (ps < 0.05); Lower left AG, right SPL, IPL activation was found during MI compared to ME (ps < 0.05).

Figure 5

Extracted brain regions associated with execution, observation, and imagination. The axial view is on the top right, while the sagittal view is on the bottom right. The color bar refers to the p-values of the correlations in the region. Moderate correlations were found over the left IPL (r = 0.39; p = 0.059), left POCG (r = 0.38, p = 0.061), and left SMG (r = 0.29, p = 0.069) during ME; over the left IPL (r = 0.46; p = 0.053), left SMG (r = 0.38; p = 0.059), and left POCG (r = 0.32; p = 0.066) during MO; and over the left SMG (r = 0.50, p = 0.059), left POCG (r = 0.41; p = 0.061), and left IPL (r = 0.20, p = 0.063) during MI.