Virtual action and real action have different impacts on comprehension of concrete verbs

Abstract

In the last decade, many results have been reported supporting the hypothesis that language has an embodied nature. According to this theory, the sensorimotor system is involved in linguistic processes such as semantic comprehension. One of the cognitive processes emerging from the interplay between action and language is motor simulation. The aim of the present study is to deepen the knowledge about the simulation of action verbs during comprehension in a virtual reality setting. We compared two experimental conditions with different motor tasks: one in which the participants ran in a virtual world by moving the joypad knob with their left hand (virtual action performed with their feet plus real action performed with the hand) and one in which they only watched a video of runners and executed an attentional task by moving the joypad knob with their left hand (no virtual action plus real action performed with the hand). In both conditions, participants had to perform a concomitant go/no-go semantic task, in which they were asked to press a button (with their right hand) when presented with a sentence containing a concrete verb, and to refrain from providing a response when the verb was abstract. Action verbs described actions performed with hand, foot, or mouth. We recorded electromyography (EMG) latencies to measure reaction times of the linguistic task. We wanted to test if the simulation occurs, whether it is triggered by the virtual or the real action, and which effect it produces (facilitation or interference). Results underlined that those who virtually ran in the environment were faster in understanding foot-action verbs; no simulation effect was found for the real action. The present findings are discussed in the light of the embodied language framework, and a hypothesis is provided that integrates our results with those in literature.

Keywords: abstract verbs; action verbs; comprehension; embodied language; virtual reality.

FIGURE 1

Tools used to explore VR. On the left: the Xbox 360 joypad; the circle indicates the knob used to walk in the virtual environment, and the dart the key pressed to give the response. On the right: the Vuzix AV920 head-mounted display (HMD).

FIGURE 2

A screenshot of the virtual park displayed in the Run condition.

FIGURE 3

A screenshot of the virtual park displayed in the Video condition.

FIGURE 4

The performances of the two groups for each type of verb. * indicates a difference statistically significant.