The role of object categories in hybrid visual and memory search

Abstract

In hybrid search, observers search for any of several possible targets in a visual display containing distracting items and, perhaps, a target. Wolfe (2012) found that response times (RTs) in such tasks increased linearly with increases in the number of items in the display. However, RT increased linearly with the log of the number of items in the memory set. In earlier work, all items in the memory set were unique instances (e.g., this apple in this pose). Typical real-world tasks involve more broadly defined sets of stimuli (e.g., any "apple" or, perhaps, "fruit"). The present experiments show how sets or categories of targets are handled in joint visual and memory search. In Experiment 1, searching for a digit among letters was not like searching for targets from a 10-item memory set, though searching for targets from an N-item memory set of arbitrary alphanumeric characters was like searching for targets from an N-item memory set of arbitrary objects. In Experiment 2, observers searched for any instance of N sets or categories held in memory. This hybrid search was harder than search for specific objects. However, memory search remained logarithmic. Experiment 3 illustrates the interaction of visual guidance and memory search when a subset of visual stimuli are drawn from a target category. Furthermore, we outline a conceptual model, supported by our results, defining the core components that would be necessary to support such categorical hybrid searches.

Figure 1

A simple model of Hybrid Search

Figure 2

RT x Set Size functions for correct present and absent trials in Experiment 1a. Error bars show +/− 1 SEM (within-observer errors, calculated using the method of (Cousineau, 2005)).

Figure 3

Results from Experiments 1a & 1b compared to results from (Wolfe, 2012). Data points represent the slope of the RT x Visual Set Size functions as a function of the memory set size on a log scale. Experiment 1a data for individual subjects are shown as black and white small circles and squares. Average Experiment 1b data are shown as outlined squares. Within-subject standard errors fall within the data points. Gray circles are data taken from Wolfe (2012). The line is a best-fit regression. Dashed lines represent 95% confidence intervals around the Wolfe (2012) data.

Figure 4

RT as a function of visual set size for Experiment 2 (Solid lines). Data from comparable conditions from (Wolfe, 2012) are plotted with black outline figures and dotted lines (Error bars, where visible, are +/− 1 SEM, within observer).

Figure 5

RT as a function of Memory Set Size. Solid lines show data from Experiment 2. Dotted lines show comparable conditions from (Wolfe, 2012). Symbols to the right show predictions of Memory Set Size 8 data. The shaded figures are based on linear extrapolation from memory set sizes 1, 2, & 4. The open figures are based on logarithmic extrapolation from log2 (memory set sizes 1, 2, & 4). Error bars, where visible, are +/− 1 within observer SEM.

Figure 6

A more elaborated model of Hybrid Search

Figure 7

For a given memory set, there are three types of items in the visual set: Those that could not possibly be targets (e.g. letters, in this case), items that have visual properties like those in the memory set but are categorically incorrect (e.g. boxing gloves are not animals), and those that are both visually and categorically appropriate and, thus, require a search of the memory set to determine their status.

Figure 8

Slopes of the functions relating RT to number of items in a specific category (target – diamond, closed-Exp 3a, open-3b, non-target – circle, or feature distinct – square) as a function of the memory set size. This is a measure of the cost of each additional distractor of a specific variety. Fig 8a shows average target present for the 8 Os with reasonable error rates. 8b shows target absent trials. Error bars are +/− 1 SEM (within-observer errors).