Attention directs actions in visual foraging

Abstract

Visual foraging tasks, where participants collect items by touching or clicking on them, have become popular for investigating visual search. They probe selective attention in multi-target contexts through naturalistic goal-directed actions, unlike the button presses used in many other paradigms. Despite their potential, such tasks had not been used to examine the interplay of attention and goal-directed actions until now, even though this topic has been extensively studied with other paradigms and has significant implications for understanding human visual behavior in the real world. In this study, we applied the visual foraging paradigm to address this gap. We found that attentional prioritization of one part in a two-part compound object is accompanied by a motor bias in the collecting action (stylus tap) toward the prioritized part. This bias combines with motor precision demands, such as aiming for stable contact points. Our findings show that action planning not only modulates the attentional landscape at large but also that attentional asymmetries (e.g., prioritizing one object part) feed back into the motor system, combining with motoric factors to refine goal-directed actions.

Keywords: Goal-directed actions; Naturalistic visual search; Selection for action; Selective attention; Visual foraging.

Fig. 1

Tablet-PC with stimuli to scale. In the example, eight targets have already been collected. One exemplary target and one distractor are shown in magnification. Exemplary photo of the interaction with the tablet-PC, adapted from Ref. [40] / CC-BY 4.0.

Fig. 2

Question and response options in the search template self-report after the foraging task. Whether the “red rectangle / green half-circle” or “green rectangle / red half-circle” elements were shown depended on which of the two versions the participants were instructed to search. Note that the appearance in the figure was slightly altered for better presentation here: Texts were translated from German to English, font size was increased, and the gray background color was replaced with white.

Fig. 3

(A) Average (over trials) inter-target times at different target counts (number of collected items) in the current study. One line per participant. (B) Average end peak factors (last ITT divided by median ITT) for the current study and Kristjánsson et al.’s 2014 study. Boxes delimit the interquartile range (IQR). Vertical bars inside the boxes represent the median. Whiskers extend to include the lowest/highest data points in a range from the first quartile minus 1.5 times the IQR to the third quartile plus 1.5 times the IQR. Points outside the range are plotted explicitly.

Fig. 4

Tap position plots for all twenty participants. The rectangles and half-circles represent the targets (normalized to the same upright orientation) and their parts with their different colors. The small gray semi-transparent dots show the individual taps from all selections from all trials. The white markings were generated with Gaussian kernel density estimation. The white dot indicates the highest density the increasingly larger contours indicate levels that contain 10%, 25%, and 50% highest density. The faint gray circular area in the background represents the radius from the object center within which taps were accepted to collect the object. The little icons in the upper right corners represent the self-reported template use by highlighting the reported part(s). Figure best viewed in color.

Fig. 5

(A) Descriptive plot of the proportion of switches between the object parts produced by each participant (stars) when foraging. The black dot with error bars indicates the average and standard error of the mean, both calculated on arcsine-square-root-transformed data and transformed back to a proportion scale. (B) The posterior distribution of the switching probability estimated for the group level. In both (A) and (B) the white background area indicates run-like behavior (repeated selections of the same object part beyond those that happen randomly) and the turquoise area indicates random switching (note that technically only the probability of 0.5 indicates truly random switching, while probabilities larger than 0.5 indicate increasingly systematic alternations between the alternatives, typically not seen in visual foraging data).

Fig. 6

(A) Posterior of the switching probability $p^{\text{switch}}$. The white area in the panel represents run-like selections of the object parts and turquoise shaded area random switching (and systematic alternating, see note in Fig. 5). (B) Circle markers show posterior means of the preference for tapping on red parts ($p^{\text{red}}$) on the participant level. Horizontal bars the 95% highest-density intervals. The triangular markers point to the template icon representing the self-reported preference. Diamond markers indicate participants who reported switching between parts. Panels (C) and (D) show the counts with which object parts in a certain color and shape were reported in the self-reports across participants (cf. triangle markers in panel B). Panels (E) and (F) show the counts of participants who preferred tapping one or the other feature (cf. circle markers in panel B, relative to 0.5). Panel (G) visualizes the proportion (model estimates) of rectangle tapping preferences for participants who reported having attended to the rectangle and those who reported having attended to the circular part. Each bar is one participant (four participants who reported switching were excluded), the error bars represent the 95% HDIs.

Fig. 7

(A) Correlation between the tapping preferences for red parts estimated from the first ($p_{\text{1st}}^{\text{red}}$) and the second ($p_{\text{2nd}}^{\text{red}}$) half of the experiment. Points indicate the marginal posterior modes and error bars the 95% highest-density intervals. The green shaded area contains the values that correspond to preferring the green object part in both experiment halves and the red shaded area shows consistent preference for the red part. (B) Estimated tapping preferences of the red object parts ($p_b^{\text{red}}$) over ten successive blocks $b$ of the experiment. The color of shaded areas indicates color of preference. Lines depict the posterior modes and the faint error bands the 95% highest-density intervals.