Model cortical association fields account for the time course and dependence on target complexity of human contour perception

Abstract

Can lateral connectivity in the primary visual cortex account for the time dependence and intrinsic task difficulty of human contour detection? To answer this question, we created a synthetic image set that prevents sole reliance on either low-level visual features or high-level context for the detection of target objects. Rendered images consist of smoothly varying, globally aligned contour fragments (amoebas) distributed among groups of randomly rotated fragments (clutter). The time course and accuracy of amoeba detection by humans was measured using a two-alternative forced choice protocol with self-reported confidence and variable image presentation time (20-200 ms), followed by an image mask optimized so as to interrupt visual processing. Measured psychometric functions were well fit by sigmoidal functions with exponential time constants of 30-91 ms, depending on amoeba complexity. Key aspects of the psychophysical experiments were accounted for by a computational network model, in which simulated responses across retinotopic arrays of orientation-selective elements were modulated by cortical association fields, represented as multiplicative kernels computed from the differences in pairwise edge statistics between target and distractor images. Comparing the experimental and the computational results suggests that each iteration of the lateral interactions takes at least [Formula: see text] ms of cortical processing time. Our results provide evidence that cortical association fields between orientation selective elements in early visual areas can account for important temporal and task-dependent aspects of the psychometric curves characterizing human contour perception, with the remaining discrepancies postulated to arise from the influence of higher cortical areas.

Figure 1. Examples of targets and distractors from the amoeba/no-amoeba image set for different .

From left to right: . Top row: Targets; amoeba complexity increases with increasing numbers of radial frequencies. Clutter was constructed by randomly rotating groups of amoeba contour fragments. Bottom row: Distractors; only clutter fragments are present.

Figure 2. ODD kernels.

Top Row: For a single short line segment oriented approximately horizontally at the center (not drawn), the co-occurrence-based support of other edges at different relative orientations and spatial locations is depicted. Axes were rotated by () from vertical to mitigate aliasing effects. The color of each edge was set proportional to its co-occurrence-based support. The color scale ranges from blue (negative values) to white (zero) to red (positive values). Left panel: Co-occurrence statistics compiled from target images. Center panel: Co-occurrence statistics compiled from distractor images. Right panel: ODD kernel, given by the difference in co-occurrence statistics between target and distractor kernels. Bottom Row: Subfields extracted from the middle of the upper left quadrant (as indicated by black boxes in the top row figures), shown on an expanded scale to better visualize the difference in co-occurrence statistics between target and distractor images. Alignment of edges in target images is mostly cocircular whereas alignment is mostly random in distractor images, accounting for the fine structure in the corresponding section of the ODD kernel.

Figure 3. The effect of lateral interactions on example images.

Left column: black and white amoeba-target image (). Right column: Gray-scale natural image (the standard computer vision test image “Lena”) after applying a hard Difference of Gaussians (DoG) filter to enhance edges. Top row: Raw retinal input. Second row: Responses of orientation-selective elements before any lateral interactions (). To aid visualization, the activity of the maximally responding orientation-selective element at each pixel location is depicted as a gray-scale intensity. Rows 3-6: Activity after iterations of the multiplicative ODD kernel, as labeled. For each iteration, activity was multiplied by the local support, computed via linear convolution of the previous output activity with the ODD kernel. Lateral interactions tended to support smooth contours, particularly those arising from amoeba segments, while suppressing clutter or background detail.

Figure 4. Histograms of total luminance in target and distractor images as a function of the number of iterations.

Red bins: Total activity histograms for all test target images. Blue bins: Total activity histograms for all test distractor images. The degree that the two distributions overlap is shown as the gray shaded area, which provides a measure of whether total luminance can be used to distinguish targets from distractors. The percentage in each shaded area shows the approximate lower bound amount of overlap of the two histograms, for comparison. Top row: Total summed activity over all retinal pixels. Little, if any bias between target and distractor images was evident in the input black and white images as there is nearly complete overlap between the distributions. Subsequent rows: Total activity histograms summed over all orientation-selective elements. Second row: Bottom-up responses prior to any lateral interactions. Third - sixth rows: Total activity histograms after - iterations of the multiplicative ODD kernel, respectively. Total summed activity became progressively more separable with additional iterations, as evinced by a decrease in the overlapping areas.

Figure 5. Psychophysical experiment schematic.

The stimulus consisted of one target image and one distractor image (randomly positioned with equal probability on the left or right), presented simultaneously for an SOA between ms and ms, followed by an optimized ms mask generated from randomly rotated groups of target and distractor segments. Subjects indicated which side contained the target object (amoeba) using a computer mouse to click along a horizontal slider bar. Clicking far to the left or right indicated strong confidence that the corresponding side contained the target; clicking close to the center indicated weak confidence. A narrow gap in the center forced subjects to choose between left and right.

Figure 6. ROC curves comparing human and model performance on the amoeba/no amoeba task.

Top two rows: ROC curves averaged over four different human test subjects using reported confidence (points). The dashed diagonal line in each plot indicates the curve corresponding to chance. Red, blue, green, black correspond to , respectively. Bottom two rows: ROC curves for model cortical association fields computed from total activity histograms.

Figure 7. A comparison of human and model performance on the 2AFC amoeba/no amoeba task.

Left: Average human performance for different SOA in milliseconds. Right: Performance of model cortical association fields for increasing numbers of iterations. Both panels: Accuracy, which is equivalent to area under the ROC curve, (error bars) fitted to single sigmoidal functions (solid lines). The four curves from top to bottom correspond to radial frequencies.

Figure 8. A comparison of ODD and simpler “Bowtie” kernel performance on the on the 2AFC, amoeba/no amoeba task plotted as a function of the number of iterations for a range of different kernel strengths.

Line width and marker size denote values on kernel strength, which was the main free parameter in the model. Kernel strength is a dimensionless constant. Black lines: ODD kernel performance. Blue lines: “Bowtie” kernel performance. Qualitative behavior was similar for both kernels, demonstrating that multiplicative lateral interactions act robustly to reinforce smooth closed contours.