Empirically grounding grounded cognition: the case of color

Abstract

Grounded cognition theories hold that the neural states involved in experiencing objects play a direct functional role in representing and accessing object knowledge from memory. However, extant data marshaled to support this view are also consistent with an opposing view that perceptuo-motor activations occur only following access to knowledge from amodal memory systems. We provide novel discriminating evidence for the functional involvement of visuo-perceptual states specifically in accessing knowledge about an object's color. We recorded event-related brain potentials (ERPs) while manipulating the visual contrast of monochromatic words ("lime") in a semantic decision task: responses were made for valid colors ("green") and locations ("kitchen") and withheld for invalid colors and locations. Low contrast delayed perceptual processing for both color and location. Critically, low contrast slowed access to color knowledge only. This finding reveals that the visual system plays a functional role in accessing object knowledge and uniquely supports grounded views of cognition.

Fig. 1

Potential outcomes in amodal and grounded architectures. Each panel depicts an amodal (A, B) or grounded (C) architecture and the processes underlying a color verification decision. From left to right, contrast-dependent low-level visual regions propagate visual form information to the fusiform gyrus (FG), where visual word form processing becomes view-invariant. Some time after the word form is processed, conceptual information becomes available to a decision-making system that can signal the motor system to execute or withhold a response in the go/nogo task. The solid lines in the right-most panel schematically depict the rate of evidence accumulation for color and location decisions and the dotted line indicates the threshold for signaling the response. In this experiment the onset latency of the N200 ERP effect provides an upper bound on the time by which this response threshold is reached. A) An amodal architecture wherein word form information is propagated eventually to amodal memory systems, where semantic access provides evidence to a decision system that signals the response. B) Identical to 1A with the exception that FG remains sensitive to the visual contrast of written words, resulting in a constant delay in low (versus high) contrast color and location decisions. C) A grounded architecture wherein word form information in FG is directly routed to nearby or overlapping FG tissue involved in color perception and also access to color knowledge. Low contrast text causes a sensory-based equivalent slowing of access to color and to location knowledge, but access to color knowledge is additionally slowed by the direct effect of low contrast on the neural resources involved in accessing color knowledge. Note that although it is not depicted here, we assume that feedback may occur at some or all levels of these architectures.

Fig. 2

Depiction of stimuli and trial timing. High and low contrast stimuli shown here are not veridical depictions of actual experimental stimuli; please see Methods for exact stimulus parameters (i.e., luminance, hue, saturation, size, etc.).

Fig. 3

Visuo-perceptual ERP components are not influenced by semantic decision type. The P2 ERP component is an obligatory neural response to visual stimuli. Low contrast delays the P2 component by about 50 ms for the first word (Figure 3A) and second word (Figure 3B) of the word pairs. Importantly, the 50 ms delay induced by low contrast (text) is not significantly different for color and location decisions.

Fig. 4

Differences between Go and NoGo ERP waveforms reveal a differential delay in access to color knowledge. ERP waveforms averaged across five electrode sites over frontal cortex (legend in bottom left corner) for high contrast trials (Panel A), low contrast trials (Panel B), and for nogo – go difference ERPs (N200 effect; Panel C). Note the delay in the low versus high contrast N200 effect in the color decisions in comparison with the location decisions (yellow shading; panel C).

Fig. 5

The N200 ERP effect is substantially more delayed by low contrast text for color decisions versus location decisions. Raster plots (outer four plots) show the results of repeated measures t-tests computed at every time point between 100 and 600 ms at 11 frontal and prefrontal scalp sites. T-tests that are statistically significant at an FDR level of 0.05 are shaded in black. Laterality (left, middle, right) is represented by the top, middle, and bottom sections of each raster plot. The onset of a reliable N200 effect is defined here as the first time point after which statistically significant t-tests occur for fifteen or more consecutive time points (60 ms window) at two or more electrode sites, and is depicted by the border between the shaded and non-shaded halves of each raster plot. Black “bar-bells” visible between the high and low contrast plots depict the difference in milliseconds between low and high contrast trials for the color decisions (left side; 204 ms) and location decisions (right side; 40 ms). Scalp maps (inner four plots) show spherical spline interpolated distributions of the grand average nogo – go ERP difference (N200 effect) between 300 and 400 ms. By this latency the frontally-distributed N200 effect is visible for both decision types under high contrast, but for location decisions only under low contrast.

Fig. 6

Differentially delayed access to color knowledge persists from a relatively early marker of semantic to behavior. Visual perception is operationalized as the peak latency of the visual P2 component during go trials, and is delayed by a constant amount in all low contrast trials. An upper bound of semantic access is operationalized as the onset of a significant N200 ERP effect (see Figure 5), and is delayed in all low contrast trials, but additionally delayed for color decisions. The differential delay persists for several hundred milliseconds and is also visible in the peak latency of the P3 ERP component on go trials (stimulus evaluation) and the mean response times for go trials (behavior).