Decoding intention: a neuroergonomic perspective

Abstract

Decoding the intentions of other people based on non-linguistic cues such as their body movement is a major requirement of many jobs. Whether it is maintaining security at an airport or negotiating with locals in a foreign country, there is a need to maximize the effectiveness of training or real-time performance in this decoding process. This review considers the potential utility of neuroergonomic solutions, and in particular, of electroencephalographic (EEG) methods for augmenting action understanding. Focus is given to body movements and hand-object interactions, where there is a rapid growth in relevant science. The interpretation of EEG-based signals is reinforced by a consideration of functional magnetic resonance imaging experiments demonstrating underlying brain mechanisms that support goal oriented action. While no EEG method is currently implemented as a practical application for enhancing the understanding of unspoken intentions, there are a number of promising approaches that merit further development.

Figure 1

The action observation network as defined by fMRI. Cortical regions demonstrating significant recruitment during the observation of a hand reaching and grasping a lever, pushing or pulling the lever, and observing the outcome (a lamp turning on or off). In this case, the actions were observed in a virtual environment. Nevertheless, the pattern of activity in the AON is similar to what is observed in movies and real, directly observed actions. When actions include information about the outcome or intention of agent, there is increased recruitment bilaterally in both frontal, parietal and posterior temporal cortex, as shown here. Data taken from a sample of 24 subjects, statistical threshold p <0.05 FDR corrected.

Figure 2

Hierarchical organization of the task of parking a car. The ultimate goal of parking a car subsumes several levels of means and goals. Completion of the ultimate goal requires completion of interim goals by way of specific actions. The parts and whole structure of conceptualizing complex goals is apparent, as is the nesting of subordinate actions within more complex actions. Imaging studies suggest that action understnading in the brain is sensitive to this nested hierarchical structure. Importantly, what is coded in the brain as an outcome and what is coded as a means to that outcome may vary depending on what level of this conceptual structure is being represented. The required level of representation depends on the context, the task at hand, and the information available to the observer at the time of testing.

Figure 3

Repetition suppression used to isolate two levels of action representation. The upper row illustrates a sequence of experimental trials in an experiment of action understanding. The images represent the final frame (and action state) in short videos that subjects observed. Within the videos, action means and outcomes were independently repeated. The second and third rows illustrate the predicted neural and fMRI BOLD responses in regions of the brain that decode means and outcomes, respectively. According to the principles of repetition suppression, regions that decode the means of an action should show greater activity on the first and last trials, in which the means of executing the action (pushing and pulling the lantern handle) are novel with respect to the previous trial. These same regions should show reduced activity on the second and third trials, in which the means are repeated. Regions that decode outcome should show greater activity on the first and second trials, in which the outcome of the action (lantern turns off or on) is novel. These regions should show decreased activity on the third and fourth trials, in which the outcome is repeated. By identifying distinct areas of the brain that respond according to unique predicted response profiles such as these, neural modules for specific aspects of action representation can be distinguished.