Reading Increases the Compositionality of Visual Word Representations

Abstract

Reading causes widespread changes in the brain, but its effect on visual word representations is unknown. Learning to read may facilitate visual processing by forming specialized detectors for longer strings or by making word responses more predictable from single letters-that is, by increasing compositionality. We provided evidence for the latter hypothesis using experiments that compared nonoverlapping groups of readers of two Indian languages (Telugu and Malayalam). Readers showed increased single-letter discrimination and decreased letter interactions for bigrams during visual search. Importantly, these interactions predicted subjects' overall reading fluency. In a separate brain-imaging experiment, we observed increased compositionality in readers, whereby responses to bigrams were more predictable from single letters. This effect was specific to the anterior lateral occipital region, where activations best matched behavior. Thus, learning to read facilitates visual processing by increasing the compositionality of visual word representations.

Keywords: neuroimaging; object recognition; open data; reading; visual search.

Fig. 1.

Malayalam and Telugu scripts. The Malayalam and Telugu languages are spoken in geographically distinct regions in India, highlighted on the map. The scripts have distinct letter shapes but share many phonemes (indicated above each letter). Only 16 example letters are shown here from each language; Telugu has 60 letters, and Malayalam has 53 letters. The full set of stimuli is shown in Section S1 in the Supplemental Material available online. Map courtesy of Free Vector Maps (https://freevectormaps.com/).

Fig. 2.

Example search array and results from Experiment 1. An example single-letter search array using Telugu letters is shown in (a). Average search time (b) is shown for readers and nonreaders of Telugu and Malayalam letters. The baseline response time is also shown for each group of subjects. Error bars depict standard errors of the mean across subjects, and asterisks indicate statistically significant differences between groups (p < .00005, sign-rank test across pairs). Pairwise search dissimilarity is shown separately for 630 pairs of (c) Telugu letters and (d) Malayalam letters, plotted for readers and nonreaders. Each point represents one search pair; an example easy and hard search pair are shown. The dotted line is the y = x line, and the solid line is the best-fitting line to the data. Asterisks indicate that the correlations were significant (p < .00005).

Fig. 3.

Example search array and results from Experiment 2. An example search array using Telugu bigrams is shown in (a). The Telugu and Malayalam letters used to create all 25 possible bigrams are shown below the search array. Average search time (b) is shown for readers and nonreaders of Telugu bigrams and Malayalam bigrams. The baseline response time (RT) is also shown for each group of subjects. Error bars depict standard errors of the mean across subjects, and asterisks indicate significant differences between groups (p < .00005). A schematic of the part-sum model is shown in (c). According to this model, the net dissimilarity (1/RT) between bigrams AB and CD can be explained using single-letter dissimilarities between letters at corresponding locations, at opposite locations in the two bigrams, and within each of the bigrams (see the text for further details). Observed bigram dissimilarity (d) is plotted against predicted bigram dissimilarity from the part-sum model for Telugu readers on Telugu bigrams. Searches with low-frequency bigrams (n = 91) and high-frequency bigrams (n = 55) are plotted separately from all other search pairs (n = 154; gray circles). Each point represents one search pair. A few example search pairs of each type are shown in the plot. The diagonal line is the y = x line. Asterisks indicate that the mean correlations were significant (p < .00005). Part-sum model parameters (averaged across 10 part relations) are shown for letter dissimilarities at corresponding locations, across locations, and within bigrams for readers and nonreaders of (e) Telugu bigrams and (f) Malayalam bigrams. Error bars indicate standard deviations. Asterisks indicate statistical significance (*p < .05, ***p < .0005, ****p < .00005 on a signed-rank test across 10 part relations between readers and nonreaders). Average search time (g) is shown for transposed-letter searches (e.g., AB among BA) and repeated-letter searches (e.g., AA vs. BB) for readers and nonreaders, averaged across Telugu and Malayalam readers. Error bars depict standard errors of the mean across subjects, and asterisks indicate significant differences between groups (p < .00005 on a rank-sum test across search times for 20 AB–BA pairs across the two languages, or across AA–BB pairs). The model equations below the graph show how smaller within-bigram terms lead to increased dissimilarity for transposed letters but not repeated letters. For transposed-letter searches, letters are identical at opposite locations, so the opposite-location terms are multiplied by zero, but the smaller within-bigram terms for readers lead to larger dissimilarities (and therefore faster searches). For repeated-letter searches, the within-bigram terms are multiplied by zero by definition, and therefore there is no benefit for readers. Partial correlation (h) is illustrated between reading fluency and each part-sum model term (after factoring out all other terms) across subjects. We used subjects’ data across multiple experiments to perform this analysis. See Section S6 in the Supplemental Material for details. The combined model is based on predicting reading fluency as a linear combination of all model terms. Error bars represent ±1 SD, and asterisks indicate significant partial correlations (*p < .05, **p < .005, ***p < .0005).

Fig. 4.

Neural correlates of reading expertise and results from Experiment 3. Regions of interest (ROIs) are shown in (a) for an example subject, showing V1 to V3, V4, lateral occipital (LO) region, visual word-form area (VWFA), and temporal gyrus (TG). Average activation levels in Telugu readers, Malayalam readers, and the combination of both are shown for known and unknown scripts, separately for (b) V1 to V3, (c) V4, (d) VWFA, (e) TG, and (f) LO. Error bars indicate ±1 SEM across subjects. Asterisks indicate significant differences between activation levels to known and unknown scripts (*p < .05, **p < .005, ***p < .0005, ****p < .00005, in a signed-rank test comparing subject-wise average activations). The correlation between neural dissimilarity and behavioral dissimilarity for bigrams (g) is shown for each ROI, separately for the known script (left) and unknown script (right). Error bars indicate standard deviation of the correlation between the group behavioral dissimilarity and ROI dissimilarity calculated repeatedly by resampling subjects with replacement across 1,000 iterations. Asterisks inside bars indicate that the correlation between group behavior and group ROI dissimilarity was significant (*p < .05, **p < .005, ***p < .0005, ****p < .00005). Asterisks above bars indicate the fraction of bootstrap samples in which the observed difference was violated (*p < .05, **p < .005). All significant comparisons are indicated.

Fig. 5.

Compositionality of neural bigram representations in Experiment 3. A schematic of the voxel-population model is shown in (a). The response of each bigram across voxels was modeled as a linear combination of the constituent letter responses. To evaluate the model fit, we calculated the correlation between observed and predicted activations for each voxel. Average model correlation across voxels (b) is presented for each of five regions of interest, separately for known and unknown scripts. The regions are V1 to V3, V4, lateral occipital (LO) region, visual word-form area (VWFA), and temporal gyrus (TG). Error bars indicate standard errors of the model correlation across subjects. The asterisk indicates a significant difference between script types (p < .05, using a signed-rank test on subject-wise model correlations between the two groups). Average model correlation in the anterior lateral occipital region (c) is shown for Telugu readers, Malayalam readers, and both groups combined, separately for known and unknown scripts. Error bars represent standard errors of the mean across subjects. Asterisks represent statistical significance, as obtained using a signed-rank test comparing average model correlations across subjects (*p < .05, **p < .005, ****p < .00005).