Dynamic Orchestration of Brains and Instruments During Free Guitar Improvisation

Abstract

Playing music in ensemble requires enhanced sensorimotor coordination and the non-verbal communication of musicians that need to coordinate their actions precisely with those of others. As shown in our previous studies on guitar duets, and also on a guitar quartet, intra- and inter-brain synchronization plays an essential role during such interaction. At the same time, sensorimotor coordination as an essential part of this interaction requires being in sync with the auditory signals coming from the played instruments. In this study, using acoustic recordings of guitar playing and electroencephalographic (EEG) recordings of brain activity from guitarists playing in duet, we aimed to explore whether the musicians' brain activity synchronized with instrument sounds produced during guitar playing. To do so, we established an analytical method based on phase synchronization between time-frequency transformed guitar signals and raw EEG signals. Given phase synchronization, or coupling between guitar and brain signals, we constructed so-called extended hyper-brain networks comprising all possible interactions between two guitars and two brains. Applying a graph-theoretical approach to these networks assessed across time, we present dynamic changes of coupling strengths or dynamic orchestration of brains and instruments during free guitar improvisation for the first time. We also show that these dynamic network topology changes are oscillatory in nature and are characterized by specific spectral peaks, indicating the temporal structure in the synchronization patterns between guitars and brains. Moreover, extended hyper-brain networks exhibit specific modular organization varying in time, and binding each time, different parts of the network into the modules, which were mostly heterogeneous (i.e., comprising signals from different instruments and brains or parts of them). This suggests that the method capturing synchronization between instruments and brains when playing music provides crucial information about the underlying mechanisms. We conclude that this method may be an indispensable tool in the investigation of social interaction, music therapy, and rehabilitation dynamics.

Keywords: EEG hyperscanning; brain-instrument coupling; extended hyper-brain networks; graph-theoretical approach; intra- and inter-brain coupling; phase synchronization; social interaction.

Figure 1

Transformation of guitar signals and calculation of the Integrative Coupling Index (ICI). (A) Raw signal of the guitar recording. (B) Wavelet transform of the guitar recording in time-frequency domain. (C) New guitar signal derived from the wavelet-transformed signal by averaging the wavelet power across the frequency bins of interest. The whole range between 50 and 2,000 Hz is presented here. (D) Wavelet transform of the new guitar signal in the time-frequency domain used for calculation of the ICI. (E) Raw EEG signal. (F) Wavelet transform of the EEG signal in time-frequency domain. (G) Time course of instantaneous phases from two signals and their phase difference. Phase of the guitar signal = green curve; phase of the EEG signal = blue curve; phase difference (Δφ) between the two signals = red curve). (H) Coding of the phase difference (–p/4 < Δφ < 0: blue stripes; 0 < Δφ < +p/4: red stripes; Δφ < –p/4 or Δφ > +p/4: green stripes = non-synchronization).

Figure 2

Extended hyper-brain network. (A) Connectivity matrix. The network consisting of 88 nodes and 7,656 edges in total includes 40 nodes (electrodes) of guitarist A's brain, 40 nodes of guitarist B's brain, four nodes of guitar A, and 4 nodes of guitar B. The four nodes in the guitars display four different frequency ranges used during signal transformation. The connection strength in the matrix is displayed by colors ranging from dark blue (threshold connectivity values) to dark red (high connectivity values). (B) Connectivity maps. The upper panel represents the connectivity within the guitars and brains, and the lower panel represents the connectivity between the guitars and brains. The strength of the nodes (sum of all out-going connections) is coded by circle size, and the strength of edges is coded by the width of the line. The different parts of the network are color-coded: guitar A, green; guitar B, yellow; guitarist A's brain, blue; guitarist B's brain, red. (C) Topological distribution of the strengths. The upper two maps represent the topological distribution of the out-strengths within the brains of the two guitarists, and the lower ones display the topological distribution of the out-strengths going from guitarist A's brain to guitarist B's brain (left) and vice versa (right). (D) Community structure of the network. The connectivity maps are presented as in (B) but nodes and corresponding edges are color-coded according to module affiliation. Note that most of the modules are heterogeneous, comprising different guitars and brains, with exception of the blue module, which is located within the brain of guitarist A.

Figure 3

Dynamic changes of strengths during a 10-s improvisation period for FC1 (1.25 Hz) in duo 1. (A) Guitar traces obtained by microphone recording: guitar A, blue; guitar B, red. (B) Dynamic changes of coupling strengths going from guitar A to the brains of both guitarists for each of the four frequency ranges of the guitar signal, which are indicated by color: low range, brown; middle range, cyan; high range, purple; whole range, yellow. (C) Dynamic changes of coupling strengths going from guitar A to the brains of both guitarists for each of the four frequency ranges of the guitar signal, which are indicated by the same colors as in (B). (D) Dynamic changes of coupling strengths going from guitar A to the guitar B (blue) and vice versa (red). (E) Dynamic changes of coupling strengths within the brains of each of the two guitarists. (F) Dynamic changes of coupling strengths going from guitarist A's brain to guitarist B's brain (blue) and vice versa (red). (G) Brain connectivity maps, community structures, and topological distribution of coupling strengths. See Figure 2 for explanations.

Figure 4

Dynamic changes of strengths during a 10-s improvisation period for FC1 (1.25 Hz) in duo 2. (A) Guitar traces obtained by microphone recording: guitar A, blue; guitar B, red. (B) Dynamic changes of coupling strengths going from guitar A to the brains of both guitarists for each of the four frequency ranges of the guitar signal, which are indicated by color: low range, brown; middle range, cyan; high range, purple; whole range, yellow. (C) Dynamic changes of coupling strengths going from guitar A to the brains of both guitarists for each of the four frequency ranges of the guitar signal, which are indicated by the same colors as in (B). (D) Dynamic changes of coupling strengths going from guitar A to guitar B (blue) and vice versa (red). (E) Dynamic changes of coupling strengths within the brains of each of the two guitarists. (F) Dynamic changes of coupling strengths going from guitarist A's brain to guitarist B's brain (blue) and vice versa (red). (G) Brain connectivity maps, community structures, and topological distribution of coupling strengths. See Figure 2 for explanations.

Figure 5

Power Spectral Density (PSD) of second order oscillations represented by strength dynamics. (A) PSD of coupling strengths between two guitars. (B) PSD of coupling strengths going from guitar A to the brains of both guitarists. (C) PSD of coupling strengths going from guitar B to the brains of both guitarists. (D) PSD of the hyper-brain coupling strengths. (E) PSD of the within-brain coupling strengths. (F) PSD of the between-brain coupling strengths. (G) Average PSD of all coupling strengths presented in A-F. For representation, PSD was averaged across 10 trails and two duos for the four FCs: FC1 (1.25 Hz), blue; FC2 (2.5 Hz), red; FC3 (5 Hz), green; and FC4 (10 Hz), yellow.