A one-man bilingual cocktail party: linguistic and non-linguistic effects on bilinguals' speech recognition in Mandarin and English

Abstract

Multilingual speakers can find speech recognition in everyday environments like restaurants and open-plan offices particularly challenging. In a world where speaking multiple languages is increasingly common, effective clinical and educational interventions will require a better understanding of how factors like multilingual contexts and listeners' language proficiency interact with adverse listening environments. For example, word and phrase recognition is facilitated when competing voices speak different languages. Is this due to a "release from masking" from lower-level acoustic differences between languages and talkers, or higher-level cognitive and linguistic factors? To address this question, we created a "one-man bilingual cocktail party" selective attention task using English and Mandarin speech from one bilingual talker to reduce low-level acoustic cues. In Experiment 1, 58 listeners more accurately recognized English targets when distracting speech was Mandarin compared to English. Bilingual Mandarin-English listeners experienced significantly more interference and intrusions from the Mandarin distractor than did English listeners, exacerbated by challenging target-to-masker ratios. In Experiment 2, 29 Mandarin-English bilingual listeners exhibited linguistic release from masking in both languages. Bilinguals experienced greater release from masking when attending to English, confirming an influence of linguistic knowledge on the "cocktail party" paradigm that is separate from primarily energetic masking effects. Effects of higher-order language processing and expertise emerge only in the most demanding target-to-masker contexts. The "one-man bilingual cocktail party" establishes a useful tool for future investigations and characterization of communication challenges in the large and growing worldwide community of Mandarin-English bilinguals.

Keywords: Bilingual; Coordinate response measure; Informational masking; Selective attention; Speech recognition.

Fig. 1

Task Schematic for Experiments 1 and 2. In each experiment, bilingual Mandarin–English or non-Mandarin-speaking English participants heard two competing speech streams over the left and right channels of their headphones. These streams either matched or mismatched in the language (English/Mandarin). Each speech stream followed a stylized pattern corresponding to “Ready go to now.” Participants listened for their call sign (e.g., “Ready, Baron!”) to identify the target stream and report the color and number it conveyed on a 6 (number) × 4 (color) response grid. In Experiment 1, both bilingual Mandarin–English and non-Mandarin-speaking English participants responded to English language targets with distracting English and Mandarin speech across two task conditions. In Experiment 2, bilingual Mandarin–English participants responded to all combinations of Mandarin and English speech assigned as target versus distractor across four task conditions

Fig. 2

Experiment 1 Mean proportion correct trials (color/number responses) as a function of target-to-masker ratio (dB). All target sentences were spoken in English; responses with Mandarin-distractor masker shown in gray, and English-distractor masker in black. Responses from English speakers without access to Mandarin shown with hollow circles; responses from bilingual speakers with access to Mandarin shown with filled circles. Error bars show 95% confidence intervals. Chance response is 4%

Fig. 3

Experiment 1 Mean proportion distractor responses (intrusions) as a function of TMR (dB), distractor language, and language experience. Symbols and error bars as in Fig. 2

Fig. 4

Experiment 2 Mean proportion correct responses as a function of TMR (in dB, x-axis), distractor language (Mandarin in gray shading, English in black shading), and target language (Mandarin in open circles, English in filled circles)

Fig. 5

Experiment 2 Mean proportion distractor responses (intrusions) as a function of TMR (in dB, x-axis), distractor language (Mandarin distractor in gray shading, English distractor in black shading), and target language (open circle Mandarin, filled circles English)