Imaging natural cognition in action

Abstract

The primary function of the human brain is arguably to optimize the results of our motor actions in an ever-changing environment. Our cognitive processes and supporting brain dynamics are inherently coupled both to our environment and to our physical structure and actions. To investigate human cognition in its most natural forms demands imaging of brain activity while participants perform naturally motivated actions and interactions within a full three-dimensional environment. Transient, distributed brain activity patterns supporting spontaneous motor actions, performed in pursuit of naturally motivated goals, may involve any or all parts of cortex and must be precisely timed at a speed faster than the speed of thought and action. Hemodynamic imaging methods give information about brain dynamics on a much slower scale, and established techniques for imaging brain dynamics in all modalities forbid participants from making natural extensive movements so as to avoid intractable movement-related artifacts. To overcome these limitations, we are developing mobile brain/body imaging (MoBI) approaches to study natural human cognition. By synchronizing lightweight, high-density electroencephalographic (EEG) recording with recordings of participant sensory experience, body and eye movements, and other physiological measures, we can apply advanced data analysis techniques to the recorded signal ensemble. This MoBI approach enables the study of human brain dynamics accompanying active human cognition in its most natural forms. Results from our studies have provided new insights into the brain dynamics supporting natural cognition and can extend theories of human cognition and its evolutionary function - to optimize the results of our behavior to meet ever-changing goals, challenges, and opportunities.

Keywords: EEG; ICA; Mobile brain imaging; Natural cognition.

Fig. 1

Examples for MoBI setups from left to right (A and B) as used in the Mobile Brain/Body Imaging Lab at the Swartz Center for Computational Neuroscience (Makeig), (C) the Human Neuromechanics Laboratory in Michigan, and (D) the Berlin Mobile Brain/Body Imaging Lab in Berlin (Gramann). (A) In a ‘conducting’ experiment, novice and expert music listeners were invited to expressively ‘conduct’ music excerpts while their movements and EEG were recorded. Here a participant with 128-channel EEG cap and full-body motion capture suit with an addition LED sensor on the middle finger of his ‘conducting’ hand (picture courtesy Dr. Grace Leslie). (B) A dart game investigation with a participant aiming at the center of the dart board and throwing a dart. Here the recording included 128 EEG electrodes, 64 electrodes measuring neck muscle activity, and 64 arm electrode, motion capture, ground force plate, video, and behavioral measures (picture courtesy Dr. Makoto Miyakoshi). (C) A gait research setup with a participant on a treadmill, 128 Channel EEG, motion capture of the lower limbs, EMG of the lower limbs, a dual band force measuring treadmill, and external input devices for manual reactions. (D) A participant wearing 128-channel EEG, 32 channels for recording neck muscle activity, plus motion capture reflectors on the head, upper torso, and finger while playing a flying sphere game.

Fig. 2

Schematic view of the Lab Streaming Layer (LSL) software framework for collecting, storing, and processing multi-modal laboratory data including data collected in MoBI experiments. LSL runs on a local area network (or, conceptually, a compute cloud network) and efficiently links data providers (physiological and/or behavioral recording systems) with data consumers (data viewer, recorder, or analysis facilities) in MoBI experiments.

Fig. 3

Architecture of the Simulation and Neuroscience Application Platform (SNAP). Users create SNAP scripts that run desired experimental protocols (top panel). SNAP component functions run on top of and interact with the core Panda3D game engine. SNAP itself runs on the open computer language Python. SNAP allows relatively simple Python scripting of a wide range of fixed or interactive task paradigms, while also supporting development and delivery of highly complex, video game-like MoBI experiment applications.

Fig. 4

A) Grand average event-related potentials (ERPs) and single subject ERPs before and after spatially filtering EEG data. Participants were fast walking on a treadmill while detecting visually presented targets. Overlapping single-subject ERP traces are shown before (light pink traces) and after (gray traces) spatially filtering and rejection of artifacts using ICA. Bold traces show the grand average ERPs at the indicated electrode locations in the fast walking condition, before (red) and after (black) removing non-brain independent component (IC) processes. Scalp maps show grand average scalp topographies of the raw (left) and the artifact-removed ERPs (right) at 400 ms. White dots indicate the locations of the indicated electrodes. B) Upper and lower rows display scalp maps from mean projections to the scalp of the indicated clusters of independent component (IC) processes. Upper row from left to right displays scalp maps of brain-based clusters with cluster centroid dipole locations located in or near the anterior cingulate cortex, right and left motor cortex, and superior parietal cortex. Middle row displays equivalent-dipole locations of IC processes (small spheres) and respective IC cluster centroids (large spheres) projected on horizontal, sagittal, and coronal views of the standard MNI brain. (Yellow) Neck-muscle ICs; (gray) eye-movement ICs; (other colors) brain-based ICs. Lower row from left to right displays scalp maps of non-brain-based clusters with cluster centroid dipole locations located in or near the neck region reflecting neck muscle activity (left splenius capitis and right sternocleidomastoid) and vertical and horizontal eye movement activity. Modified from Gramann et al. (2010).