The nature of letter crowding as revealed by first- and second-order classification images

Abstract

Visual crowding refers to the marked inability to identify an otherwise perfectly identifiable object when it is flanked by other objects. Crowding places a significant limit on form vision in the visual periphery; its mechanism is, however, unknown. Building on the method of signal-clamped classification images (Tjan & Nandy, 2006), we developed a series of first- and second-order classification-image techniques to investigate the nature of crowding without presupposing any model of crowding. Using an "o" versus "x" letter-identification task, we found that (1) crowding significantly reduced the contrast of first-order classification images, although it did not alter the shape of the classification images; (2) response errors during crowding were strongly correlated with the spatial structures of the flankers that resembled those of the erroneously perceived targets; (3) crowding had no systematic effect on intrinsic spatial uncertainty of an observer nor did it suppress feature detection; and (4) analysis of the second-order classification images revealed that crowding reduced the amount of valid features used by the visual system and, at the same time, increased the amount of invalid features used. Our findings strongly support the feature-mislocalization or source-confusion hypothesis as one of the proximal contributors of crowding. Our data also agree with the inappropriate feature-integration account with the requirement that feature integration be a competitive process. However, the feature-masking account and a front-end version of the spatial attention account of crowding are not supported by our data.

Figure 1

Flanker analysis procedure. The masking noise plus the flankers are classified akin to the conventional first-order classification images, and a z test is performed for each category against the expected mean for that category under the null hypothesis that neither the flankers nor the noise affects the behavioral response. The z scores are plotted on a color-coded map to reveal statistically significant structures of the flankers and noise that are correlated with the subject’s response. The regions where the target and the flankers were presented are demarcated by the bounding boxes of the letters.

Figure 2

Procedure to calculate the second-order feature maps. Correlation coefficients between each pixel in an ROI_opt and the corresponding pixel in an offset region (ROI_off) are accumulated over all residual noise fields for a particular error-response category and are plotted in terms of Fisher’s Z to facilitate comparison across conditions and subjects on a color-coded map. This map reveals the second-order statistics of the features that comprise an observer’s perceptual template. The center of the map represents any pixel in ROI_opt; a “hot” spot or a positive correlation at an offset (Δx, Δy) from the center indicates that the observer is biased toward a particular response when two pixels separated by (Δx, Δy) have the same contrast polarity; a “cold” spot or a negative correlation at an offset (Δx, Δy) from the center indicates that the response is partly driven by two pixels separated by (Δx, Δy) with opposite contrast polarity.

Figure 3

Rationale behind the estimation of the ROI_opt. Regions smaller than the optimal ROI do not capture all the correlation in the data and, hence, have lower significance (higher p value). Regions larger than the optimal also have lower significance due to the inclusion of uncorrelated noise.

Figure 4

(A) Examples of stimuli used in the four experimental conditions. (B) Timing of stimuli presentation: (1) fixation beep immediately followed by a fixation screen for 500 ms, (2) stimulus presentation for 250 ms, (3) subject response period (variable) with positive feedback beep for correct trials, and (4) 500 ms delay before onset of next trial.

Figure 5

(A) Log thresholds in SNR (contrast energy E divided by noise spectral density N) for the four experimental conditions. There is a significant increase in E/N in the periphery-flanked condition as compared with the periphery-unflanked condition. There is also a small but significant opposite effect in the fovea. Note that the standard error bars are smaller than the plot symbol. (B) First-order classification images. The raw images have been filtered by a Gaussian kernel (space constant = 1.4 pixels) to aid visualization. The numbers indicate the number of trials for the corresponding stimulus–response category. For the fovea conditions with small letter size, only the central portion of the classification images is magnified and shown.

Figure 6

Flanker analysis results: (A) z-score maps (Z_OX, upper right box; Z_XO, lower left box) thresholded at α = .05 (|Z| > 1.96) indicate the presence (positive contrast) or absence (negative contrast) of significant features that bias an observer’s response; (B) t-test maps, also thresholded at α = .05, directly compare the error trials of the two response categories. The “hot” regions indicate features that bias toward “o,” whereas the “cold” regions are indicative of feature that bias toward “x.” For the fovea conditions, only the central portion of the classification images is magnified and shown.

Figure 7

Estimated ROI_opt or feature utilization zones. The significance (mean of log p values) of the different candidate ROIs are color coded, and these color-coded regions are superimposed in ascending order of significance; the most significant regions (demarcated by the blue dotted lines) represent the optimal ROIs. The positions of the target and the flankers are superimposed to give a sense of the extent of the utilization zones.

Figure 8

(A) Geometric mean of the p values in the ROI_opt, which is a measure of the amount of features present in the ROI. (B) Horizontal extent of the feature utilization zones.

Figure 9

(A) Second-order feature maps obtained from an ideal-observer model using letter stimuli (shown above the corresponding feature map) identical to that used in the foveal experimental conditions. White and black contour lines demarcate the positive and negative correlation zones at rZ = ±1.0. (|rZ| > 1.96 corresponds to an α level of .05.) The axes of the maps are in units of x-height. (B) Feature maps obtained using the peripheral stimuli.

Figure 10

Second-order feature maps obtained from the human observer’s data overlaid with positive (demarcated by the black contour lines) and negative correlation zones (white contour lines) from the corresponding ideal-observer feature maps (Figure 9). The axes of the maps are in units of x-height. |rZ| > 1.96 corresponds to an α level of .05.

Figure 11

Quality of match (Q_m), feature utilization (U), and feature validity (V) of the human feature maps (Figure 10) as compared with the ideal-observer maps (Figure 9). The error bars represent standard error estimates obtained by bootstrapping on the human feature maps.