Turning visual search time on its head

Abstract

Our everyday visual experience frequently involves searching for objects in clutter. Why are some searches easy and others hard? It is generally believed that the time taken to find a target increases as it becomes similar to its surrounding distractors. Here, I show that while this is qualitatively true, the exact relationship is in fact not linear. In a simple search experiment, when subjects searched for a bar differing in orientation from its distractors, search time was inversely proportional to the angular difference in orientation. Thus, rather than taking search reaction time (RT) to be a measure of target-distractor similarity, we can literally turn search time on its head (i.e. take its reciprocal 1/RT) to obtain a measure of search dissimilarity that varies linearly over a large range of target-distractor differences. I show that this dissimilarity measure has the properties of a distance metric, and report two interesting insights come from this measure: First, for a large number of searches, search asymmetries are relatively rare and when they do occur, differ by a fixed distance. Second, search distances can be used to elucidate object representations that underlie search - for example, these representations are roughly invariant to three-dimensional view. Finally, search distance has a straightforward interpretation in the context of accumulator models of search, where it is proportional to the discriminative signal that is integrated to produce a response. This is consistent with recent studies that have linked this distance to neuronal discriminability in visual cortex. Thus, while search time remains the more direct measure of visual search, its reciprocal also has the potential for interesting and novel insights.

Figure 1

Subjects performed oddball visual search for a bar that differed in orientation from multiple identical distractors (Experiment 1). (A) Visual search reaction time (RT) plotted against difference in orientation between the target and distractors. Observed reaction times (dots) are shown with error bars representing standard error of the mean (s.e.m). These data were fit using a linear fit (dashed lines), an inverse fit (i.e. a linear fit on 1/RT versus orientation difference; thick line) and a sigmoid fit to 1/RT (thin line). (B) Reciprocal of search time, 1/RT, plotted against the orientation difference (dots with error bars representing s.e.m). These data were fit using a linear fit on orientation differences below 30 degrees (thick line) and using a sigmoid function (thin line).

Figure 2

Subjects performed multiple visual searches involving all possible pairs of 48 images with each image in a pair as target or distractor (Experiment 2). For each triplet of images (A, B, C) – depicted schematically above the plot – the pairwise dissimilarity measurements (1/RT) were denoted by d_AB, d_BC and d_AC. If 1/RT is a distance metric, it should satisfy the triangle inequality. In other words, each of these distances (plotted on the y-axis, depicted as d_AC) must be smaller than the sum of the other two (plotted on the x-axis, depicted as d_AB + d_BC). The inset shows the histogram of the difference (d_AB+ d_BC – d_AC) across all triplets. Triplets with a difference less than zero (in the inset plot) or triplets that fall below the y = x line (in the scatter plot) violate the triangle inequality. Such violations occurred only in 143 of 51,888 triplets, i.e. in 0.2% of all triplets.

Figure 3

Asymmetry analysis. (A) A total of 69 of 1128 image pairs in Experiment 2 showed a statistically significant asymmetry. For each of these image pairs, the discriminability (1/RT) of the search involving the easy target is plotted against the discriminability of the hard target. The data was fit using a straight line with slope of nearly 1. (B) The same data represented using search times (RT). For the same 69 image pairs, the search times for the easy target are plotted against the search times for the hard target. The data suggest that the size of the asymmetry increases in proportion to the average search time. (C) Illustration of how asymmetries differing by a constant amount in Δd (as shown in A) might give rise to RT differences that increase with the mean (as shown in B) – see text for details.

Figure 4

Visualization of search distances using multi-dimensional scaling (Experiment 2). Multi-dimensional scaling was performed on all 276 pair-wise search distances (1/RT) between 24 images (6 objects x 4 views each), to find the best-fitting two-dimensional coordinates such that their distances match the observed pair-wise distances. The resulting plot shows clustering of objects by view as well as mirror confusion between left & right views.