An Alternative Explanation for Difficulties with Speech in Background Talkers: Abnormal Fusion of Vowels Across Fundamental Frequency and Ears

Abstract

Normal-hearing (NH) listeners use frequency cues, such as fundamental frequency (voice pitch), to segregate sounds into discrete auditory streams. However, many hearing-impaired (HI) individuals have abnormally broad binaural pitch fusion which leads to fusion and averaging of the original monaural pitches into the same stream instead of segregating the two streams (Oh and Reiss, 2017) and may similarly lead to fusion and averaging of speech streams across ears. In this study, using dichotic speech stimuli, we examined the relationship between speech fusion and vowel identification. Dichotic vowel perception was measured in NH and HI listeners, with across-ear fundamental frequency differences varied. Synthetic vowels /i/, /u/, /a/, and /ae/ were generated with three fundamental frequencies (F₀) of 106.9, 151.2, and 201.8 Hz and presented dichotically through headphones. For HI listeners, stimuli were shaped according to NAL-NL2 prescriptive targets. Although the dichotic vowels presented were always different across ears, listeners were not informed that there were no single vowel trials and could identify one vowel or two different vowels on each trial. When there was no F₀ difference between the ears, both NH and HI listeners were more likely to fuse the vowels and identify only one vowel. As ΔF₀ increased, NH listeners increased the percentage of two-vowel responses, but HI listeners were more likely to continue to fuse vowels even with large ΔF₀. Binaural tone fusion range was significantly correlated with vowel fusion rates in both NH and HI listeners. Confusion patterns with dichotic vowels differed from those seen with concurrent monaural vowels, suggesting different mechanisms behind the errors. Together, the findings suggest that broad fusion leads to spectral blending across ears, even for different ΔF₀, and may hinder the stream segregation and understanding of speech in the presence of competing talkers.

Keywords: binaural fusion; concurrent vowel; dichotic; hearing loss.

Fig. 1

Hearing thresholds for the subjects in this study. The left and right panels show the left and right ears, respectively. Normal-hearing group average audiograms are shown as thick gray lines and circles, with the gray shaded area indicating the minimum and maximum thresholds. Hearing-impaired individual audiograms are shown as thin lines with symbols, and the group average audiograms are shown as black lines. (Color online)

Fig. 2

Touchscreen response options. Subjects were instructed to select either one or two vowels depending on how many they heard (highlighted in a different color when selected), then select “Done.” Subjects also had the option to repeat the vowel presentation if needed

Fig. 3

Binaural fusion ranges of the subjects in the study. Fusion ranges are shown for reference tones of 2000 Hz for HI listeners (solid black circles) and NH listeners (solid gray circles), as well as 1000 Hz for HI listeners (open triangles). HI listeners show broader fusion ranges as well as greater variation in fusion ranges compared with NH listeners

Fig. 4

Relationship of percent single-vowel responses and vowel recognition scores to the fundamental frequency difference between dichotic vowels. a Percent single-vowel responses versus ΔF₀ for HI listeners. b Percent single-vowel responses versus ΔF₀ for NH listeners. c Percent correct recognition of both vowels versus ΔF₀ for HI listeners. d Percent correct recognition of both vowels versus ΔF₀ for NH listeners. e Percent correct recognition of at least one vowel versus ΔF₀ for HI listeners. f Percent correct recognition of at least one vowel versus ΔF₀ for NH listeners. Thin colored lines with markers show individual data, and thick black dashed lines show group average trends. Fusion decreases and performance increases with ΔF₀ for both groups, with NH listeners showing more benefit of ΔF₀. For legend for HI subjects, see Fig. 1

Fig. 5

Relationship of vowel recognition score to vowel fusion, indicated by percentage of single vowel responses. Percentage correct recognition of one vowel in the pair given that a single vowel was selected is plotted versus the percentage of single-vowel responses for HI listeners (black squares) and NH listeners (gray squares). The solid line indicates a linear fit of the NH and HI data pooled together; the correlation is significant. Note that the y-axis starts at 50 %, and the chance level for when one vowel is selected

Fig. 6

Relationship of vowel fusion to tone fusion range. The percentage of single vowel responses is significantly correlated with the fusion range measured using tones for all listeners pooled together and for NH listeners alone (gray circles), but not for HI listeners alone (black circles). Linear fits and R² values are shown for NH, HI, and pooled (all) data as dotted, dashed, and dash-dot lines, respectively

Fig. 7

Relationship of percent single-vowel responses and vowel recognition scores to the fundamental frequency difference between monaural concurrent vowels. Only data from the left ear is shown; similar results were observed for left and right ears. a Percent single-vowel responses versus ΔF₀ for HI listeners. b Percent single-vowel responses versus ΔF₀ for NH listeners. c Percent correct recognition of both vowels versus ΔF₀ for HI listeners. d Percent correct recognition of both vowels versus ΔF₀ for NH listeners. e Percent correct recognition of at least one vowel versus ΔF₀ for HI listeners. f Percent correct recognition of at least one vowel versus ΔF₀ for NH listeners. Thin colored lines with markers show individual data, and thick black dashed lines show group average trends. Fusion decreases and performance increases with ΔF₀ for both groups

Fig. 8

Group average dichotic vowel confusion matrices. Left and right panels show NH and HI groups, respectively. Top and bottom panels show confusions for ΔF₀ = 0 and ΔF₀ = 11 semitones, respectively. Each column indicates a specific vowel pair presented, while rows indicate specific response combinations, including single vowel responses. The numbers and darkness of shading in each cell indicate the number of times a response combination (specified by row location) was selected in response to a vowel pair (indicated by column location). For ΔF₀ = 0, single-vowel responses in the first 4 rows predominate and include selections of single vowel responses not in the original vowel pair, such as /ae/ for /a/ + /i/. For ΔF₀ = 11, more responses on the diagonal, indicating correct double vowel responses, are seen, especially for NH listeners

Fig. 9

Example individual dichotic vowel confusion matrices for ΔF₀ = 11 semitones, for NH and HI listeners with narrow, medium, and broad binaural tone fusion. The NH listener with narrow fusion (top left panel) has a high proportion of both vowels correct, indicated by responses on the diagonal. NH and HI listeners with medium binaural tone fusion (right panels) have confusion matrices with similar proportions of correct responses for both vowels (on the diagonal) and fused single-vowel responses (top 4 rows). One HI listener with broad binaural tone fusion mainly fused all vowel pairs into single vowels (top 4 rows), even with a ΔF₀ of 11 semitones

Fig. 10

Comparison of dichotic versus monaural concurrent vowel confusion matrices for NH62, a listener with narrow tone fusion, at ΔF₀ = 0 semitones. This listener had a high proportion of single-vowel responses (top 4 rows) in the dichotic condition (top) as well as in the monaural left ear and monaural right ear conditions (left and right, respectively). However, note that the specific vowel confusions differed between the dichotic condition and both monaural conditions (cells highlighted in thick red borders)

Fig. 11

Comparison of dichotic versus monaural concurrent vowel confusion matrices for NH78, a listener with somewhat narrow tone fusion, at ΔF₀ = 0 semitones. Plotted as in Fig. 10. Again, note that specific vowel confusions differed between the dichotic and both monaural conditions

Fig. 12

Comparison of dichotic versus monaural concurrent vowel confusion matrices for HI25, a listener with broad tone fusion, at ΔF₀ = 0 semitones. Again, note that specific vowel confusions differed between the dichotic (top) and the left ear monaural condition (bottom; the right ear was not tested in this subject)

Fig. 13

Simple model of how the perception of /u/ and /ae/ could arise from monaural and dichotic /a/ + /i/, respectively. a–d The filtered spectra of the original vowels /a/, /ae/, /i/, and /u/, with circles indicating the formant peaks selected automatically by estimation of the first two local maxima. e Monaural perception of /a/ + /i/ is modeled as the addition of the two original signals in the time domain, followed by the same spectral filtering as the original vowels. This spectrum predicts two formant peaks most similar to the peaks observed for /u/ (D). f Dichotic perception of /a/ + /i/ is modeled as the linear averaging of the spectra of /a/ (A) and /i/ (C), with wider smoothing windows. The resulting spectrum predicts two formant peaks most similar to the peaks observed for /ae/ (b)