Temporal alignment of electrocorticographic recordings for upper limb movement

Abstract

The detection of movement-related components of the brain activity is useful in the design of brain-machine interfaces. A common approach is to classify the brain activity into a number of templates or states. To find these templates, the neural responses are averaged over each movement task. For averaging to be effective, one must assume that the neural components occur at identical times over repeated trials. However, complex arm movements such as reaching and grasping are prone to cross-trial variability due to the way movements are performed. Typically initiation time, duration of movement and movement speed are variable even as a subject tries to reproduce the same task identically across trials. Therefore, movement-related neural activity will tend to occur at different times across the trials. Due to this mismatch, the averaging of neural activity will not bring into salience movement-related components. To address this problem, we present a method of alignment that accounts for the variabilities in the way the movements are conducted. In this study, arm speed was used to align neural activity. Four subjects had electrocorticographic (ECoG) electrodes implanted over their primary motor cortex and were asked to perform reaching and retrieving tasks using the upper limb contralateral to the site of electrode implantation. The arm speeds were aligned using a non-linear transformation of the temporal axes resulting in average spectrograms with superior visualization of movement-related neural activity when compared to averaging without alignment.

Keywords: ECoG; arm movement; dynamic time warping; electrocorticography; kinematics; movement classification.

Figure 1

Location of implanted ECoG contacts with respect to head representation. Contacts of the first strip of electrodes are labeled 0–3 from distal to proximal and contacts of the second strip are similarly indexed 4–7. Primary motor cortex is colored in velvet and the primary sensory cortex is colored in amber. The area associated with the hand representation is marked in green.

Figure 2

Illustration of non-linear alignment or time-warping. The time registration path defines how one time point in one trial maps to another time point in a different trial. The difference between two velocities becomes minimal when the time axis is transformed using the time-registration path.

Figure 3

Superimposed arm velocity for 25 trials when Subject 1 was reaching for a target. The subject's arm returned to its initial location after a pause of several seconds having reached the target. The trials were aligned to the initial movement onset and are marked as t = 0. The variation in the duration of movement is clearly visible.

Figure 4

(A–C) Data from Subject #1. (A) The average spectrogram and the associated EMG activity as calculated through conventional averaging. Epochs were aligned with respect to onset of the reaching task with no time-warping used. Movement onset is denoted by dash dotted line (“-.”). The average spectrogram was normalized with respect to the baseline, which is defined as the power between 1 and 2 s prior to movement onset. (B) Same as (A) but with data aligned to movement onset of the retrieval task. (C) Same as (A,B) but with non-linear warping of the time axis prior to averaging. (D–F) Shows the same for Subject #2.

Figure 5

Same as Figure 4 but with only ECoG data from Subject #3 (A–C) and Subject #4 (D–F).

Figure 6

Comparing the warping of spectrograms (left panels) with the spectrograms of signals warped in the time-domain (right panels). Percentile changes in spectral density of the ECoG activity is shown with respect to the rest period. (A/B,C/D,E/F,G/H) shows results for Subject #1, #2, #3, #4 respectively. The distortion is evident in the spectrograms after warping is carried out on the time-domain signal.