Bringing the real world into the fMRI scanner: repetition effects for pictures versus real objects

Abstract

Our understanding of the neural underpinnings of perception is largely built upon studies employing 2-dimensional (2D) planar images. Here we used slow event-related functional imaging in humans to examine whether neural populations show a characteristic repetition-related change in haemodynamic response for real-world 3-dimensional (3D) objects, an effect commonly observed using 2D images. As expected, trials involving 2D pictures of objects produced robust repetition effects within classic object-selective cortical regions along the ventral and dorsal visual processing streams. Surprisingly, however, repetition effects were weak, if not absent on trials involving the 3D objects. These results suggest that the neural mechanisms involved in processing real objects may therefore be distinct from those that arise when we encounter a 2D representation of the same items. These preliminary results suggest the need for further research with ecologically valid stimuli in other imaging designs to broaden our understanding of the neural mechanisms underlying human vision.

Figure 1. Experimental setup, stimulus items and fMRI trial sequence.

(a) Participants lay supine in the MR scanner with the head supported within the lower portion (6 elements) of an inclined 12-channel head coil. A 4-channel flex coil was positioned over the front of the head. The head coil was tilted forward by ∼30^o to enable direct viewing of stimuli. 2D pictures and 3D object stimuli (illustrated above) were mounted by the experimenter on a turntable positioned over the waist. The experiment was conducted in complete darkness and all trials recorded using an infra-red camera. Stimulus presentation duration was controlled by an LED (illuminator). Participants were asked to identify the objects presented on each trial while maintaining fixation on a single red LED positioned just above the stimulus plane. (b) Six sets of 5 stimulus exemplar objects were used in the fMRI-A experiment (30 in total). A different set of stimulus items was used in each run to prevent cross-adaptation. 3D stimuli from ‘Set 1' are depicted in (a) and (c). (c) Example trial sequence. Each stimulus item was presented for 500ms within a 3 sec inter-stimulus interval. Stimuli for each upcoming trial were positioned on the turntable during the 20 sec inter-trial interval.

Figure 2. Timecourse of fMRI signals within LO and pFS for 2D-pictures and 3D-objects.

Data are group results (n = 13). (a–b) Upper panels show responses within LO for Different versus Repeat 2D-pictures (left) or real 3D-objects (right). (c–d) Lower panels show responses within pFS for Different versus Repeat 2D-pictures (left) or real 3D-objects (right). (—) Trials in which a different stimulus appeared on each trial (Different condition). (---) Trials in which identical stimuli appeared (Repeat condition).

Figure 3. Repetition effects for 2D pictures and 3D objects within LO and pFS.

The magnitude of repetition effects for each stimulus type within each region was quantified using an Adaptation Index (AI). The AI represents differences in responses between Repeat and Different conditions relative to the overall fMRI response, thereby providing a measure of repetition effects scaled according to activation levels for each stimulus in each ROI. Positive index values reflect higher responses on Different than Repeat trials; negative values indicate the reverse pattern and values around zero indicate a lack of repetition effects. To provide meaningful data interpretation in a within-subjects design, error bars for the difference scores are based on the 95% confidence intervals, which indicate whether or not the average difference was significantly greater than zero (with probabilities equal to those from the t-test).

Figure 4. Group functional activation for the contrast [2D-Different > 2D-Repeat] in the whole-brain voxel-wise analysis.

Activation is overlaid on the inflated cortical surface of a representative observer. Widespread repetition-based changes in activation for pictures of objects were observed across temporal and parietal cortex. Conversely, no such activation changes were identified for real 3D objects [i.e., using the contrast 3D-Different > 3D-Repeat] at the same threshold. Dorsal surface (far left), Right Hemisphere (top middle), Left Hemisphere (lower middle) and Ventral Surface (far right). Sulci are represented in dark grey and gyri in light grey.

Figure 5. Loci of additional group-based region of interest (ROI) analyses displayed on the inflated cortex of a representative subject.

The cortex is illustrated from a posterior-ventral viewpoint. Group-based region of interest analyses were conducted at the marked loci in each hemisphere (see Supplementary Table 1). (A–I): Sites within occipital cortex, intraparietal sulcus, inferior temporal gyrus and premotor cortex in which second-order disparity-selective neurons are thought to extract and process 3D depth structure from stereo. Points (H–I) lie anterior to the central sulcus and are not visible from the above viewpoint. (J–M): Topographically organized areas within the intraparietal sulcus (IPS) areas 1–4, as reported by Konen & Kastner. Using a variety of 2D greyscale picture stimuli these authors report significant adaptation effects within IPS1 and IPS2, but not more dorsally within IPS 3 and IPS4. (K) Loci correspond to LOtv, located along the ventro-lateral bank of the temporal lobe. Area LOtv is selective for both visual and haptic object properties and is argued to support abstract 3D shape representations. (A) V3A complex (1); (B) V3A complex 2; (C) ITG; (D) VIPS/V7; (E) POIPS; (F) DIPSM; (G) DIPSA; (H) dPrCS; (I) vPrCS; (J) IPS1; (K) IPS2; (L) IPS3; (M) IPS4; (N) LO_TV. (See also Table S1.)