Speech, movement, and gaze behaviours during dyadic conversation in noise

Abstract

How do people have conversations in noise and make themselves understood? While many previous studies have investigated speaking and listening in isolation, this study focuses on the behaviour of pairs of individuals in an ecologically valid context. Specifically, we report the fine-grained dynamics of natural conversation between interlocutors of varying hearing ability (n = 30), addressing how different levels of background noise affect speech, movement, and gaze behaviours. We found that as noise increased, people spoke louder and moved closer together, although these behaviours provided relatively small acoustic benefit (0.32 dB speech level increase per 1 dB noise increase). We also found that increased noise led to shorter utterances and increased gaze to the speaker's mouth. Surprisingly, interlocutors did not make use of potentially beneficial head orientations. While participants were able to sustain conversation in noise of up to 72 dB, changes in conversation structure suggested increased difficulty at 78 dB, with a significant decrease in turn-taking success. Understanding these natural conversation behaviours could inform broader models of interpersonal communication, and be applied to the development of new communication technologies. Furthermore, comparing these findings with those from isolation paradigms demonstrates the importance of investigating social processes in ecologically valid multi-person situations.

Figure 1

Experimental set-up (example of a non-participating individual). Panel a shows the participant setup within the sound attenuated room, showing the loudspeakers (N) presenting noise throughout each trial. Panel b shows the equipment setup including motion tracking crown, eye-tracking glasses and microphone. Panel c shows an example of the noise levels (54–78 dB in 6 dB increments) as a function of time during an example conversation trial.

Figure 2

Speech adjustments by noise level. Panel a shows mean speech level by noise level. Panel b shows utterance duration by noise level, and panel c shows inter-speaker pause duration by noise level. Panel d shows proportion of time for individual speech, overlapping speech, and silence, by noise level. All error bars show 95% within-subject confidence intervals.

Figure 3

Movement adjustments by noise level. Panel a shows interpersonal distance by noise level. Panel b shows head (yaw) angle means, and panel c shows head (yaw) angle standard deviations, during periods of talking and listening by noise level. All error bars show 95% within-subject confidence intervals.

Figure 4

Gaze adjustments by noise level. Panel a shows proportion of listening time spent oriented toward the eye region, the mouth region, and the sum of the two, by noise level. Error bars show 95% within-subject confidence intervals. Panel b shows an example gaze pattern, with darker areas indicating more gaze time to illustrate how the gaze data was split into eye and mouth regions.