Auditory short-term memory activation during score reading

Abstract

Performing music on the basis of reading a score requires reading ahead of what is being played in order to anticipate the necessary actions to produce the notes. Score reading thus not only involves the decoding of a visual score and the comparison to the auditory feedback, but also short-term storage of the musical information due to the delay of the auditory feedback during reading ahead. This study investigates the mechanisms of encoding of musical information in short-term memory during such a complicated procedure. There were three parts in this study. First, professional musicians participated in an electroencephalographic (EEG) experiment to study the slow wave potentials during a time interval of short-term memory storage in a situation that requires cross-modal translation and short-term storage of visual material to be compared with delayed auditory material, as it is the case in music score reading. This delayed visual-to-auditory matching task was compared with delayed visual-visual and auditory-auditory matching tasks in terms of EEG topography and voltage amplitudes. Second, an additional behavioural experiment was performed to determine which type of distractor would be the most interfering with the score reading-like task. Third, the self-reported strategies of the participants were also analyzed. All three parts of this study point towards the same conclusion according to which during music score reading, the musician most likely first translates the visual score into an auditory cue, probably starting around 700 or 1300 ms, ready for storage and delayed comparison with the auditory feedback.

Figure 1. Experimental procedure of the EEG experiment.

Overview of the different conditions in the EEG experiment. All stimuli pairs consisted of simultaneously presented auditory and visual stimuli, but the task differed between the conditions. A/V: simultaneously presented auditory and visual stimulus to be compared, V-A: visual stimulus of the 1^st pair and auditory stimulus of the 2^nd pair to be compared, A-A: auditory stimulus of the 1^st pair and auditory stimulus of the 2^nd pair to be compared, V-V: visual stimulus of the 1^st pair and visual stimulus of the 2^nd pair to be compared. The V-A condition was also used in the behavioural task, with and without distracters presented in the time period of 0.2–1.6 s.

Figure 2. Global field potentials and voltage distributions at 1200–1600 ms.

A. Global field potentials of the conditions V-A, A-A, and V-V (with short-term memory storage) compared to A/V (without short-term memory storage). The time period of 1200–1600 ms shows statistical difference in all 3 comparisons. B. Voltage maps of all conditions at time points 1250 ms, 1400 ms, and 1550 ms. There were no statistically significant differences between any of the conditions that involved short-term memory (V-A, A-A, and V-V) in whole scalp-GFP amplitude at any point of time.

Figure 3. Potential differences between the cross-modal and unimodal conditions.

Top row: Topographic comparison between V-A and A-A, and V-A and V-V (significant differences shown in black for all electrodes over time). During the time period 1200–1600 ms (red rectangle), V-A only showed differences with V-V. Brain responses during V-A were more negative than during Vis-Vis in that time period. Scalps in the middle show the location of the electrodes that were different during that period for at least 5% of the time. For the comparison of V-A with V-V, the differences were located in left-frontal and occipital areas. Grey rectangle: Event-related potentials of electrodes selected for display because of their central location in the two areas of significant difference.

Figure 4. Reaction Times in behavioural experiment.

Reaction times of all participants in the score-reading-like V-A condition with and without distractors. Only auditory dyads were significantly interfering the performance of the task compared to when no distracters where presented. DA: auditory dyads, DV: visual dyads, IA: auditory interval names, IV: visual interval names, NO: no distractors.