The effect of visual cues on difficulty ratings for segregation of musical streams in listeners with impaired hearing

Abstract

Background: Enjoyment of music is an important part of life that may be degraded for people with hearing impairments, especially those using cochlear implants. The ability to follow separate lines of melody is an important factor in music appreciation. This ability relies on effective auditory streaming, which is much reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues could reduce the subjective difficulty of segregating a melody from interleaved background notes in normally hearing listeners, those using hearing aids, and those using cochlear implants.

Methodology/principal findings: Normally hearing listeners (N = 20), hearing aid users (N = 10), and cochlear implant users (N = 11) were asked to rate the difficulty of segregating a repeating four-note melody from random interleaved distracter notes. The pitch of the background notes was gradually increased or decreased throughout blocks, providing a range of difficulty from easy (with a large pitch separation between melody and distracter) to impossible (with the melody and distracter completely overlapping). Visual cues were provided on half the blocks, and difficulty ratings for blocks with and without visual cues were compared between groups. Visual cues reduced the subjective difficulty of extracting the melody from the distracter notes for normally hearing listeners and cochlear implant users, but not hearing aid users.

Conclusion/significance: Simple visual cues may improve the ability of cochlear implant users to segregate lines of music, thus potentially increasing their enjoyment of music. More research is needed to determine what type of acoustic cues to encode visually in order to optimise the benefits they may provide.

Figure 1. Audiograms for HA group.

Best-ear un-aided audiogram results from the HA group. Hearing thresholds are given in dB HL. The fundamental frequency of the melody notes is shown by the black triangles on the x-axis.

Figure 2. Visual display.

The simple 4-note melody (G, C, A, D, midinotes 67, 72, 69, 74) depicted on the stave used as the visual display. Each melody note turned red as it played. The scale to the right repeated the participants' response in real time, so they did not have to look away from the screen to gauge their response.

Figure 3. Electrodogram for melody notes.

An “electrodogram” showing the stimulation across electrodes (on the y-axis) over time, as the 4-note melody is repeated three times. The first note of the melody starts at time 0. The electrodogram was generated by RFStatistics software (Hearing CRC, Melbourne).

Figure 4. Task design.

Decreasing (DEC: upper panel) and increasing (INC: lower panel) blocks are shown. Melody notes (black/dark dots) play continuously. Distracter notes (red/light dots) are interleaved with the melody notes, and are selected from a range of 12 consecutive midinotes (an octave). The distracter note range is increased or decreased by one midinote per level, for 20 levels. Within each level, the melody is repeated 20 times (a single presentation is shown here).

Figure 5. Difficulty ratings across distracter note levels.

Top panel: Difficulty ratings (+/- SEM), averaged across INC and DEC blocks, as a function of distracter note level, with visual cues provided (red triangles) and with no visual cues (black squares). Bottom panel: the reduction in difficulty provided by the visual cue.

Figure 6. Mean difficulty ratings.

Mean difficulty ratings across all distracter separation levels. Significant differences (Tukey HSD test) between groups and conditions are indicated with horizontal bars.