Impact of early visual experience on later usage of color cues

Abstract

Human visual recognition is remarkably robust to chromatic changes. In this work, we provide a potential account of the roots of this resilience based on observations with 10 congenitally blind children who gained sight late in life. Several months or years following their sight-restoring surgeries, the removal of color cues markedly reduced their recognition performance, whereas age-matched normally sighted children showed no such decrement. This finding may be explained by the greater-than-neonatal maturity of the late-sighted children's color system at sight onset, inducing overly strong reliance on chromatic cues. Simulations with deep neural networks corroborate this hypothesis. These findings highlight the adaptive significance of typical developmental trajectories and provide guidelines for enhancing machine vision systems.

Figure 1.. Results of experiments with late-sighted individuals.

A. Sample stimuli for the object naming experiment in the color (left) and grayscale condition (right). B. Naming results of Prakash children, depicting individual participant data on the ‘Color > Gray benefit’ measure. C. Group means of the ‘Color > Gray benefit’ measure for Prakash children and blur-matched controls. Error bars depict the standard error. Result of t-test: *P < 0.05, **P < 0.01, ***P < 0.001, n.s., not significant. D. Exemplar stimuli for the color sensitivity experiment. E. Results of the color sensitivity test, displayed as a function of color and depicted for Prakash patients at four different time points (post-op 1: ca. 2 days after surgery; post-op 2: ca. 7 days after surgery; post-op 3: ca. 30 days after surgery, and post-op 4: ca. 6 months after surgery) as well as controls wearing 20/200 and 20/500 blur goggles. F&G. Results of the color sensitivity test, when displayed as a function of (F) luminance intensity and (G) delta (the difference in hue between the two discs). Error bars in panels E-G depict the standard error.

Figure 2.. Performance-based analysis of different network instances.

A. Color and grayscale classification performance of networks trained on the ImageNet database using our four different regimens. B. Classification performance of different networks tested on the ImageNet database when rotating the color wheel by gradually shifting the hue content of the original image. C&D. Color classification performances of networks trained on the ImageNet database, evaluated on the subset of classes depicting food (C) and fruit (D) items. E. Relationship between class-specific performance gains of the G2C model over the G2G model, and the homogeneity of mean hues across the different instances of a given class (see Supplementary Methods for details). F. 10-fold cross-validated classification performances of the four networks trained on the FaceScrub database. Error bars depict the standard error. G. 10-fold cross-validated classification performances of networks trained and tested on the FaceScrub database when rotating the color wheel. The shaded area depicts the standard error.

Figure 3.. Representational analysis of different network instances.

A. Depiction of the 96 individual receptive fields of G, C, G2C, and C2G networks trained on the ImageNet database, sorted by their colorfulness (see Supplementary Methods for details). B. Comparison of the colorfulness of individual filters when transitioning from the first to the second stage of training, depicted separately for the transition of G to G2C as well as C to C2G. C&D. Depiction of exemplar synthesized images (eliciting maximal unit activation) from the fourth convolutional (C) and last fully-connected (D) layer, for our four typical training regimens. E&F. Performance (in % correct) of n = 39 online participants in classifying synthesized images from the G2G, G2C, and C2C models into one of the three super-categories ‘Animals’, ‘Food’, or ‘Objects’ (E) and classifying them into the correct super-category as well as the correct basic-level category (F). Result of t-test: *P < 0.05, **P < 0.01, ***P < 0.001, n.s., not significant.