Regularity extraction from non-adjacent sounds

Abstract

The regular behavior of sound sources helps us to make sense of the auditory environment. Regular patterns may, for instance, convey information on the identity of a sound source (such as the acoustic signature of a train moving on the rails). Yet typically, this signature overlaps in time with signals emitted from other sound sources. It is generally assumed that auditory regularity extraction cannot operate upon this mixture of signals because it only finds regularities between adjacent sounds. In this view, the auditory environment would be grouped into separate entities by means of readily available acoustic cues such as separation in frequency and location. Regularity extraction processes would then operate upon the resulting groups. Our new experimental evidence challenges this view. We presented two interleaved sound sequences which overlapped in frequency range and shared all acoustic parameters. The sequences only differed in their underlying regular patterns. We inserted deviants into one of the sequences to probe whether the regularity was extracted. In the first experiment, we found that these deviants elicited the mismatch negativity (MMN) component. Thus the auditory system was able to find the regularity between the non-adjacent sounds. Regularity extraction was not influenced by sequence cohesiveness as manipulated by the relative duration of tones and silent inter-tone-intervals. In the second experiment, we showed that a regularity connecting non-adjacent sounds was discovered only when the intervening sequence also contained a regular pattern, but not when the intervening sounds were randomly varying. This suggests that separate regular patterns are available to the auditory system as a cue for identifying signals coming from distinct sound sources. Thus auditory regularity extraction is not necessarily confined to a processing stage after initial sound grouping, but may precede grouping when other acoustic cues are unavailable.

Keywords: auditory object formation; auditory processing; implicit learning; integration; mismatch negativity (MMN); non-adjacent dependencies; segregation; sound grouping.

Figure 1

Design of Experiment 1. Repetitive tones (black) were interspersed with tones whose frequencies followed a sine-wave pattern (orange). The frequency ranges of the two sets of tones overlapped in three experimental conditions (A–C), and were separated in a control condition (D). In all conditions, stimulus-onset asynchrony was 150 ms (i.e., 300 ms separately within each sequence). The experimental conditions differed in the length of the individual tones (50/90/130 ms). Violations of the repetition regularity were inserted to probe by MMN elicitation whether the auditory system detected this regularity despite the intervening tones.

Figure 2

Results of Experiment 1: passive listening. Group-average (N = 15) ERPs recorded at the Fz electrode for the repetitive standard tones (black line) and for the deviant tones violating the repetition regularity (green line), together with the deviant-minus-standard difference waveforms (red line). All ERPs are shown with average mastoid reference. Each panel (A–D) presents one of the four conditions. In all conditions, significant deviance-related activity was elicited in the latency range of the MMN component (120–160 ms from deviation onset).

Figure 3

Results of Experiment 1: passive listening, difference waveforms. Group-average (N = 15) deviant-minus-standard ERP difference waves in each of the four conditions for the whole electrode set. All ERPs are shown with nose reference. The latency range of the MMN component (120–160 ms from deviation onset) is marked in gray.

Figure 4

Results of Experiment 1: active deviance detection. Group-average (N = 15) ERPs recorded at the Fz electrode for the repetitive standard tones (black line) and the deviant tones violating the repetition regularity (green line), together with the deviant-minus-standard difference waveforms (red line). All ERPs are shown with average mastoid reference. Each panel (A–D) presents one of the four conditions. In all conditions, significant deviance-related activity was elicited in the latency range of the MMN component (120–160 ms from deviation onset). Note that the amplitude scale differs from that of Figures 2 and 3.

Figure 5

Design of Experiment 2. The “overlapping 130 ms” condition of Experiment 1 was repeated (A) and complemented by a control condition in which the tones of the sine-wave envelope appeared in random order (B). Again, violations of the repetition regularity were inserted to probe whether the auditory system detected the regularity across the intervening sounds.

Figure 6

Results of Experiment 2. Group-average (N = 16) ERPs recorded at the Fz electrode for the repetitive standard tones (black line) and the deviant tones violating the repetition regularity (green line), together with the deviant-minus-standard difference waveforms (red line). All ERPs are shown with average mastoid reference. In the intervening-regular condition (A), significant deviance-related activity was elicited in the latency range of the MMN component (135–175 ms from deviation onset). A significant negative difference in the same latency range was also observed in the intervening-random condition (B), but with significantly smaller amplitude.

Figure 7

Refractoriness control analysis for Experiment 2. Upper panels: Group-average (N = 16) ERPs recorded at the Fz electrode for the repetitive standard tones (black line), deviant tones violating the repetition regularity (green line), and intervening tone glides matching the deviant frequency (orange line). All ERPs are shown with average mastoid reference. Lower panels: Mean ERP amplitudes in the latency range of the MMN component (135–175 ms from deviation onset). In the intervening-regular condition [(A), left column], the intervening tone glides do not seem to be processed differently from the standard tones, arguing against a contribution of refractoriness to the negativity elicited by the deviants in the MMN latency range. In the intervening-random condition [(B), right column], there are early differences between standard and intervening tones which, however, do not translate into the MMN latency range.