Hand Motions Reveal Attentional Status and Subliminal Semantic Processing: A Mouse-Tracking Technique

Abstract

Theories of embodied cognition suggest that hand motions and cognition are closely interconnected. An emerging technique of tracking how participants move a computer mouse (i.e., the mouse-tracking technique) has shown advantages over the traditional response time measurement to detect implicit cognitive conflicts. Previous research suggests that attention is essential for subliminal processing to take place at a semantic level. However, this assumption is challenged by evidence showing the presence of subliminal semantic processing in the near-absence of attention. The inconsistency of evidence could stem from the insufficient sensitivity in the response time measurement. Therefore, we examined the role of attention in subliminal semantic processing by analyzing participants' hand motions using the mouse-tracking technique. The results suggest that subliminal semantic processing is not only enhanced by attention but also occurs when attention is disrupted, challenging the necessity of facilitated top-down attention for subliminal semantic processing, as claimed by a number of studies. In addition, by manipulating the color of attentional cues, our experiment shows that the cue color per se could influence participants' response patterns. Overall, the current study suggests that attentional status and subliminal semantic processing can be reliably revealed by temporal-spatial features extracted from cursor motion trajectories.

Keywords: area under the curve; attention; congruency effects; cursor motion; mouse-tracking; subliminal semantic processing.

Figure 1

An illustration of the “congruency effect”. The participants’ task was to judge whether the target number was larger or smaller than 5. Before presenting the target, a priming digit was displayed briefly for 29 ms, sandwiched by masks to make the priming digit subliminal. The response time was shorter for congruent trials than incongruent trials, which is the “congruency effect”.

Figure 2

An illustration of the area under the curve (AUC). In each trial, the AUC (shaded area) is measured as the number of pixels enclosed by the dashed direct line connecting the starting and ending points and the actual cursor curve that goes over the direct path toward the unselected option. Any area that goes over the direct path toward the selected choice is subtracted as negative AUC. The cursor always starts from the center of the “START” button and ends where participants click one of the two options.

Figure 3

The procedure in the green and red conditions. The two conditions differed only in the cue color.

Figure 4

The average AUCs of congruent and incongruent trials are shown for cued and uncued conditions in the green and red conditions, respectively. The AUCs were measured by amounts of pixels on the screen.

Figure 5

Average trajectories in the green condition (a) and the red condition (b), with the cued condition on the left panel and the uncued condition on the right panel. Trajectories in light blue represent congruent trials while those in light brown represent incongruent trials. All trajectories start from the lower-middle location and travel to one of the two options either on the upper-left corner (e.g., “Small”) or on the upper-right corner (e.g., “Large”) of the screen, depending on participants’ choices. The congruency effect is illustrated by the distance between the congruent and the incongruent trajectories toward the same ending location, which corresponds to the difference in average AUCs between congruent and incongruent trials. The X and Y axes denote amounts of pixels on the screen.