Alive and grasping: stable and rapid semantic access to an object category but not object graspability

Abstract

How quickly do different kinds of conceptual knowledge become available following visual word perception? Resolving this question will inform neural and computational theories of visual word recognition and semantic memory use. We measured real-time brain activity using event-related brain potentials (ERPs) during a go/nogo task to determine the upper limit by which category-related knowledge (living/nonliving) and action-related knowledge (graspable/ungraspable) must have been accessed to influence a downstream decision process. We find that decision processes can be influenced by the living/nonliving distinction by 160ms after stimulus onset whereas information about (one-hand) graspability is not available before 300ms. We also provide evidence that rapid access to category-related knowledge occurs for all items, not just a subset of living, nonliving, graspable, or ungraspable ones, and for all participants regardless of their response speed. The latency of the N200 nogo effect by contrast is sensitive to decision speed. We propose a tentative hypothesis of the neural mechanisms underlying semantic access and a subsequent decision process.

Figure 1

Experiment 1 and 2 ERPs for the category (living/nonliving) decision (Panel A), graspability (graspable/ungraspable) decision (Panel B), corresponding NoGo – Go difference waves (Panel C) and go response times (Panel D). Between 150 and 200 ms in both experiments, no-go ERPs were significantly more positive than go ERPs for category but not graspability judgments. Between 200 and 600 ms in Experiment 1, no-go ERPs were significantly more negative than go ERPs for graspability but not category judgments, and no significant difference between go and no-go trials was found in Experiment 2. As shown in the far right column, response times for go trials were significantly faster when the go/no-go decision criterion was contingent upon object category versus graspability in both experiments, and overall mean response times were substantially slower in Experiment 2.

Figure 2

Time course of global signal strength of the NoGo – Go ERP difference in Experiment 2. Root-mean square (RMS) curves for each decision criterion are computed across two sets of electrodes corresponding to anterior and posterior/central scalp sites. An early peak for object category decisions is visible in the anterior RMS curves (top row) between 150 and 200 ms, whereas no such peak occurs for graspability decisions. After 300 ms, the relatively larger N2 effect begins to develop in both conditions. Posterior RMS curves (bottom row) reveal essentially no change from baseline in either response condition until a relatively late effect begins to develop in both conditions after about 350 ms.

Figure 3

Mass univariate statistical analysis of the time course of ERP differences for each decision criterion in Experiment 2. Raster plots convey the results of repeated-measures t-tests at every 4 ms interval between 140 and 600 ms at all 26 scalp sites for category (living/nonliving) judgments (top row) and graspability (graspable/ungraspable) judgments (bottom row). Laterality (left, middle, right) is represented by the top, middle, and bottom sections of each plot. Within each section the electrode sites are listed in an anterior-to-posterior progression. False discovery rate (FDR) was controlled with an adaptive linear step-up procedure, and t-tests that are significant at an FDR level of 0.05 are marked in white (positive difference wave) or black (negative difference wave) and non-significant tests are shown in gray. Beginning after 152 ms in the object category difference wave, the difference reaches statistical significance for 8 consecutive t-tests at the midline prefrontal site. In contrast, an early frontal effect is absent in the graspability difference waves, where a run of 8 consecutive significant t-tests reflecting the N2 effect does not begin until about 340 ms at more posterior scalp sites.

Figure 4

A). Experiment 2 NoGo – Go ERPs at three prefrontal electrodes. The early effect in the category (living/nonliving) decisions is most pronounced between 150 and 200 ms, during which no difference is present for the graspability (graspable/ungraspable) decisions. B) Distributions of mean grand average potentials between 150 and 200 ms according to spherical spline interpolation. Small black circles surround the three prefrontal sites shown in A). The prefrontal maximum of the category ERP difference is clearly visible, and no such effect is visible in the graspability scalp map.

Figure 5

Experiment 2 ERPs for living thing items (Panel A), nonliving thing items (Panel B), corresponding NoGo – Go difference waves (Panel C) and response times (Panel D). Rapid access to category knowledge was found for nonliving thing objects such vehicles, utensils, or buildings as well as living thing objects such as fruits, trees, and animals. Although the NoGo – Go difference peaks later and is more pronounced for nonliving thing items, within the 150 to 200 ms time window item type (living thing vs. nonliving thing items) did not interact with the decision criterion. During the N2 window, the magnitude of the difference for both decision contingencies is larger for trials including living but not nonliving thing items and item type does not interact with decision, nor does decision type differ independently from item type.

Figure 6

Experiment 2 ERPs for graspable items (Panel A), ungraspable items (Panel B), corresponding NoGo – Go difference waves (Panel C) and response times (Panel D). Rapid access to object category knowledge was not confined to generally smaller graspable objects such as fruits, small animals, tools, and utensils, and was not confined to generally larger objects such as trees, large mammals, vehicles, and buildings. Within the 150 to 200 ms time window, item type did not interact with decision category. During the N2 window the magnitude of the differences for graspable versus ungraspable items appear somewhat different for graspability decision and almost identical for category decisions, but neither comparison reached statistical significance.

Figure 7

Experiment 2 ERPs and difference waves for fast responders (N = 13) and slow responders (N = 13). The mean go response times for fast and slow responders (median split) for each decision type are shown in D); fast responders have a 250 ms advantage over slow responders. Despite this marked difference in behavior, the NoGo – Go difference for category (living/nonliving) decisions is clearly visible in both groups between 150 and 200 ms.

Figure 8

Results of electrophysiology and MEG studies relevant to the time course of information access in word and object recognition. Entries above the timeline correspond to experiments that employed object stimuli (line drawings, photos, digitally-rendered images) and entries below the timeline correspond to experiments that employed visual word stimuli. The left-hand border of each entry depicts a rough estimate of the onset latencies of experimental effects—namely that the listed type of information is differentiated in real-time dependent measures of brain activity. Each entry is based on at least one reference (rightmost legend corresponds to references below), and represents a temporal region rather than an attempt to establish a specific onset. Italicized entries signify effects obtained during tasks requiring an overt behavioral marker of stimulus recognition such as those used in the current experiments. Non-italicized entries represent effects that were obtained in tasks that did not require an overt marker of stimulus recognition, and could be driven in part by bottom-up activation of low-level visual properties. The effects found in the current study are bolded. References Words (w1) Assadollahi & Pulvermüller, 2003; Dambacher et al., 2006; Hauk, Davis, Ford, Pulvermüller, & Marslen-Wilson, 2006a; Hauk, Patterson, Woollam, Watling, Pulvermüller, & Rogers, 2006b; Sereno, et al., 1998. (w2) Chan, et al., 2011. (w3) Amsel, 2011; Moscoso Del Prado Martin, et al., 2006. (w4) Hauk, et al., 2006a; Sauseng et al., 2004. (w5) Current study; Hauk et al., 2012. (w6) Hauk et al., 2012. (w7) Amsel, 2011; Kounios et al., 2009; Rabovsky et al., 2012. (w8) Müller & Hagoort, 2006. (w9) Current study. Objects (o1) Liu et al., 2009; VanRullen & Thorpe, 2001. (o2) Clarke et al., 2012. (o3) Allison et al., 1999; Nobre et al., 1994; Schendan et al., 1998. (o4) Vogel & Luck, 2000. (o5) VanRullen & Thorpe, 2001. (o6) Proverbio et al., 2011.