Associations Between Ancillary Body Movements and Acoustic Parameters of Pitch, Dynamics and Timbre in Clarinet Playing

Abstract

When playing an instrument, there are two main categories of body movements: instrumental movements, which are necessary for the sound production, and ancillary movements, which are associated with individual musical intentions and expressions. In this study, the particular purpose of ancillary movements of clarinet player was investigated especially in respect to how these movements were related to the musical structure of the piece and to specific audio parameters. 3D motion capture data of 19 clarinet players performing the same piece were analyzed regarding common motion patterns during the performance and in accordance with acoustic features related to pitch, dynamics (RMS energy) and timbre (spectral centroid and flux). A focus of the body movements was on the arms and the knees. The results showed that there were certain motion patterns performed by the players depending on specific musical structures. When playing a melodic part, the players often did so by bending their knees. At musical transitions, however, the knees were mainly stretched. Similarly, arm movements were more pronounced during playing melodious parts. At transitions, the arms were put closer to the torso. Considering the connection with the acoustics, a larger range of knee motions was correlated with a larger variation of the timbre. Moreover, at specific moments during the performance, when some players strongly bent their knees or lifted the arms, the RMS energy of the signal was significantly higher. The correlations of the body movements and the acoustic features showed that some players synchronized their movements with particular audio parameters more than others did. In summary, the ancillary movements of the clarinetists pursued both musical expressive intentions and physiologically necessary movements and tended to be performed with individual differences in terms of visual and auditory expression.

Keywords: body movements; expression; motion capture; music information retrieval; signal analysis.

Figure 1

Motion types of clarinetists found in Weiss et al. (2018).

Figure 2

Graphs of the visual movements of the knees and the arms (top row: schematic of the pitch; second and third row: normalized motion of the angle; fourth and fifth row: velocities in 1/50s; gray lines: individual values of the players; thick black line: mean value across all players; error bar: standard error of the mean).

Figure 3

Graphs of the four acoustic parameters (top row: schematic of the pitch; gray lines: individual values of the players; thick black line: mean value across all players; error bar: standard error of the mean).

Figure 4

Schematic of the multimodal analysis of motion peaks with sound parameters by the example of the knee movements.

Figure 5

Mean values of the acoustic parameters by motion type (Abbreviations of the motion types see Figure 1; error bar: standard error of the mean).

Figure 6

Mean values of the RMS energy, the spectral centroid and the spectral flux by motion type (Abbreviations of the motion types see Figure 1) and movement peaks (maximum: high angular peaks; minimum: low angular peaks; error bar: standard error of the mean; ^*p < 0.05).