When Gesture "Takes Over": Speech-Embedded Nonverbal Depictions in Multimodal Interaction

Abstract

The framework of depicting put forward by Clark (2016) offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: "speech-embedded nonverbal depictions," namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance.

Keywords: depiction; embedding; gesture; iconicity; multimodality.

FIGURE 1

Depiction in (1).

FIGURE 2

Depictions in (6).

FIGURE 3

Depictions in (7).

FIGURE 4

Depiction in (8).

FIGURE 5

Continuum of information contribution from non-depictive speech and depictive signals.

FIGURE 6

Depictions in (9).

FIGURE 7

Depiction in (10).

FIGURE 8

Depiction in (12).

FIGURE 9

Depiction in (13).

FIGURE 10

Depiction in (18).

FIGURE 11

Depiction in (19).

FIGURE 12

Depiction in (20).

FIGURE 13

Depiction in (21).

FIGURE 14

Depiction in (22).

FIGURE 15

Depiction in (23).

FIGURE 16

Depictions in (25).

FIGURE 17

Depictions in (27).

APPENDIX

Screenshot of annotation in ELAN (of Example 27).