Iconic Words Are Associated With Iconic Gestures

Abstract

Iconicity ratings studies have established that there are many English words which native speakers judge as "iconic," that is, as sounding like what they mean. Here, we explore whether these iconic English words are more likely to be accompanied by iconic gestures. We report a large-scale quantitative study comparing the gesture rate of words rated as high in iconicity (e.g., swoosh, puffy, crispy) to those rated as low in iconicity (e.g., ordain, rejoin, grateful), balancing for perceptual strength, part-of-speech, and syllable length. Five thousand seven hundred and twenty-five tokens from the TV News Archive were coded for whether speakers produced a gesture with the word, and whether the gesture was iconic. The results show that high iconicity words have a higher overall gesture rate (69%) than low iconicity words (56%): specifically, high iconicity words have a higher iconic gesture rate (24% vs. 11%). This effect is more pronounced among verbs than adjectives, which we hypothesize may be due to the dynamic nature of verbs. We also find that this result persists when controlling for perceptual and action strength ratings, suggesting that word-level iconicity is a more important predictor than sensorimotor strength of whether a speaker will use an iconic gesture. We find that some high iconicity words are more likely to occur with iconic gestures when they come with markers of syntactic isolation, suggesting that morphosyntactic behavior is also relevant to iconic gesture production. Our findings demonstrate that iconicity in spoken communication is inherently multimodal, manifesting in both speech and gesture simultaneously, and that iconicity is often psychologically active when speakers use conventionalized iconic words.

Keywords: Big data; Cognition; Gesture; Iconicity; Multimodal communication; Quantitative methods; Sensorimotor semantics; Sensory linguistics.

Fig. 1

Bar plot of log SUBTLEX frequency (Brysbaert & New, 2009) of words rated to be low (left) and high (right) in the iconicity ratings from Winter et al. (2023).

Fig. 2

Example of a biphasic iconic enactment gesture. (a) “it's getting …” The speaker's open hands are held apart, palms facing each other. His left hand is at hip level, palm facing upward, and his right hand is at chest level, palm facing downward. The superimposed arrow indicates the path of movement. (b) “… squished now.” The speaker moves his right hand down toward his left hand in two strokes. It lands slightly above his left hand, at around hip level. Bold italics indicate speech that co‐occurs with the stroke phase of the respective gesture.

Fig. 3

Example of a metaphorical iconic gesture. (a) “the audience was something to be…” The speaker's right hand starts at shoulder level in a fist, palm facing downward. His left hand is at stomach level, open, palm facing upward. The superimposed arrow indicates the path of movement. (b) “… outwitted .” His right hand, in a fist, moves downward and lands in his left hand, which remains open. Bold italics indicate speech that co‐occurs with the stroke phase of the respective gesture.

Fig. 4

Proportion of tokens with gestures (iconic or not) for each word. The numbers at the top of each bar indicate the number of eligible tokens each proportion is based on. Colors indicate whether the word is high or low in iconicity.

Fig. 5

Proportion of tokens with iconic gestures for each word. Of the words with a proportion of 0, five were low iconicity (absent, acquaint, ordain, prevail, sullen), and five were high iconicity (barking, gooey, plump, saggy, snap).

Fig. 6

Proportion of tokens with noniconic gestures for each word.

Fig. 7

Posterior distributions for the coefficients of the word type effect (high in iconicity vs. low in iconicity), controlling for word frequency, in separate models for (a) any gestures, (b) iconic gestures only, and (c) noniconic gestures only.

Fig. 8

High iconicity verbs show a much higher proportion of iconic gestures than low iconicity verbs, with little difference for adjectives. Squares represent posterior means, and error bars are 95% credible intervals taken from our main mixed logistic regression model amended with the part‐of‐speech interaction.

Fig. 9

Example of an iconic enactment gesture, where the speaker mimics using a sledgehammer to make a banging noise. (a) Pre‐stroke and preparing the gesture, the speaker's hands are formed in two vertical fists, right hand above the left hand, as though holding a two‐handed tool. They begin at hip level, and he raises them. (b) “ bang‐ ” The speaker's fists are at head height, now side‐by‐side, and, performing the stroke of the gesture, he brings them back down vertically. (c) “ ‐ing away.” The speaker's hands stop at hip level. Bold italics indicate speech that co‐occurs with the stroke phase of the respective gesture. The superimposed arrow indicates the path of movement.

Fig. 10

Example of an iconic depiction gesture, where the speaker traces the path of movement of an object. (a) Pre‐stroke, the speaker's open hands are held out in front of him, at about hip level. (b) “wherever you…” Pre‐stroke, the speaker prepares the gesture by raising both hands and bringing them inward, still in an open shape, to chest level. (c) “… put it ” In the stroke of the gesture, the speaker moves both hands, retaining the open hand shape, left and downward, stopping at stomach level. Bold italics indicate speech that co‐occurs with the stroke phase of the respective gesture. The superimposed arrow indicates the path of movement.