Infants exploit vowels to label objects and actions from continuous audiovisual stimuli

Abstract

Before the 6-months of age, infants succeed to learn words associated with objects and actions when the words are presented isolated or embedded in short utterances. It remains unclear whether such type of learning occurs from fluent audiovisual stimuli, although in natural environments the fluent audiovisual contexts are the default. In 4 experiments, we evaluated if 8-month-old infants could learn word-action and word-object associations from fluent audiovisual streams when the words conveyed either vowel or consonant harmony, two phonological cues that benefit word learning near 6 and 12 months of age, respectively. We found that infants learned both types of words, but only when the words contained vowel harmony. Because object- and action-words have been conceived as rudimentary representations of nouns and verbs, our results suggest that vowels contribute to shape the initial steps of the learning of lexical categories in preverbal infants.

Figure 1

Experimental protocol. In (a), we illustrate the pictures and videos used in the familiarization phase in all experiments. In the videos, the vertical and horizontal arrows indicate up-and-down and left-to-right head gestures, respectively. The words used in each experiment are written below the images, in dark blue for Experiment 1, in cyan for Experiment 2, in dark purple for Experiment 3, and in magenta for Experiment 4. The words in bold were cued either with vocalic or consonantal harmony and were tested at the end of the familiarization phase. In (b) and (c), we show the structure of the test trials evaluating the learning of the object- and action-words, respectively. Each trial began with a central audiovisual attractor, followed by the auditory presentation of a word simultaneously displayed with two empty squares, one at each side of the screen. Then, the same word was repeated twice, separated by 1 s of silence, and synchronically displayed with the presentation of 2 head gesture’s videos, one by each side in (b) or with 2 face’s pictures, one per side in (c). We recorded the infants' visual behavior by using the eye-tracking technique.

Figure 2

Window time for the analysis in each experiment. Left panels illustrate the time window when the mean gaze computed across all infants remained lateralized in (a) for Experiment 1, (c) for Experiment 2, (e) for Experiment 3, and (g) for Experiment 4. The black lines plot the proportion of lateralized gazes computed across all infants, from 500 ms before the presentation of the lateralized images to the end of the trial. The shadow indicates the standard deviation. The trial events inside the analysis window appear below the left panels. The y-axis encompasses from 0, when the gaze fell into the center of the screen, to 1 when was lateralized regardless of the side and correctness. The red lines depict the P value at each point of the same time window and show the period when the infants' gaze shifted from the center to the lateralized images and remained there (see Methods). The horizontal dotted line indicates P value < 0.01. The gray arrows indicate the onset of the time window we used for the analysis, which extended to 4700 ms in all experiments. The right panels show the accuracy for Experiment 1 in (b), Experiment 2 in (d), Experiment 3 in (f), and Experiment 4 in (h), illustrated as the mean TLT-p, TLT-acc, LF-p, and LF-acc. In the x-axis, 0.5 indicates no preference or mean gaze at chance level, values greater than 0.5 indicate a visual preference towards the correct image, while those below 0.5 a preference towards the incorrect image. The asterisks indicate a significant difference from the chance at P < 0.05.