Hierarchical computation of 3D motion across macaque areas MT and FST

Abstract

Computing behaviorally relevant representations of three-dimensional (3D) motion from two-dimensional (2D) retinal signals is critical for survival. To ascertain where and how the primate visual system performs this computation, we recorded from the macaque middle temporal (MT) area and its downstream target, the fundus of the superior temporal sulcus (area FST). Area MT is a key site of 2D motion processing, but its role in 3D motion processing is controversial. The functions of FST remain highly underexplored. To distinguish representations of 3D motion from those of 2D retinal motion, we contrast responses to multiple motion cues during a motion discrimination task. The results reveal a hierarchical transformation whereby many FST but not MT neurons are selective for 3D motion. Modeling results further show how generalized, cue-invariant representations of 3D motion in FST may be created by selectively integrating the output of 2D motion selective MT neurons.

Keywords: CP: Neuroscience; binocular vision; fundus of the superior temporal sulcus; macaque; middle temporal area; motion in depth; object motion; perspective; stereopsis.

Figure 1.. Three-dimensional-motion stimuli, discrimination task, and predictions

(A) The stimulus was a 3D volume with dots that moved toward (cyan arrow) or away (orange arrow) from the monkey’s cyclopean eye. (B) On each trial, the monkey fixated for 300 ms before a 3D motion stimulus appeared for 1 s. The perceived direction of motion was indicated by a downward (“toward”) or upward (“away”) saccade. Stimuli were always presented within the neuron’s receptive field (RF). (C) Screen-projected vector fields depicting the left- and right-eye signals for each cue condition (combined, black; left-eye perspective, blue; right-eye perspective, green; stereoscopic, magenta). Note that the 3D motion stimuli produced opposite net 2D retinal motions in the two eyes. (D) Predicted responses for 2D retinal-motion selective neurons. Left- and right-eye perspective cue responses are negatively correlated. Combined-cue and stereoscopic cue responses follow the dominant eye responses. (E) Predicted responses for 3D motion selective neurons. Preferences are the same across cue conditions.

Figure 2.. Localization of MT and FST

(A) Sagittal MRI sections from monkey J (left) and monkey C (right) showing MRI-based estimates of MT (orange), FST (purple), and the dorsal medial superior temporal (MSTd) area (teal) for reference. Blue and green dots show approximate recording locations in MT and FST, respectively. Recording sites were projected along the medial-lateral axis onto individual sections. A schematic of a laminar probe with four tetrodes and example spike waveforms is shown to the right. (B) MT neurons (left) had smaller RFs, whose area increased at a slower rate with eccentricity compared with FST neurons (right). Type II regression lines (solid black) and identity lines (dashed black) are shown for reference. Points are colored according to the anterior-posterior (A-P) location of the recording site (normalized for each monkey). The representation of the visual field in MT progressed from posterior (larger peripheral RFs; cyan) to anterior (smaller foveal RFs; magenta). A coarse reversal occurred in FST, where posterior RFs were more foveal (magenta) and anterior RFs were more peripheral (yellow). (C) Distribution of visual response latencies in MT (orange) and FST (purple). See also Figure S1.

Figure 3.. Example MT and FST responses to 3D motion

(A–C) Example MT neurons whose responses imply 2D retinal-motion selectivity (cf., Figure 1D). In (C), note that weak ocular dominance was associated with attenuated combined-cue and stereoscopic cue peak responses, consistent with motion opponency. Asymmetry index (AI) values for each cue condition are listed as insets. (D) A rare MT neuron whose responses were consistent with 3D selectivity (cf., Figure 1E). (E–G) Example FST neurons whose responses imply 3D motion selectivity. (H) An FST neuron whose responses imply 2D retinal-motion selectivity. See also Figure S2.

Figure 4.. Classification of 2D and 3D motion selectivity based on responses to eye-specific perspective cues

(A) Classification of motion selectivity in MT. Statistical boundaries demarcate regions defining neurons as 2D retinal-motion selective (red region and points), 3D motion selective (blue region and points), or unclassified (white region with black points). Most MT neurons were 2D retinal-motion selective. The diagonal histogram shows the distribution of classifications. (B) Same as (A), but for FST. Approximately equal proportions of FST neurons were selective for 2D and 3D motion. Note the bimodal distribution of classifications. See also Figures S3 and S4.

Figure 5.. Linear decoding of 3D motion direction from MT and FST

(A) Decoding performance with 2D (top) and 3D (bottom) selective MT neurons as a function of motion coherence. Decoders were trained on the combined-cue responses and tested on the combined-cue (black), stereoscopic cue (magenta), dominant eye perspective cue (yellow), and non-dominant eye perspective cue (cyan) responses. Error bars are 95% confidence intervals. (B) Same as (A), but for FST. Generalized, cue-invariant decoding of 3D motion was achieved only with 3D motion selective FST neurons. See also Figure S5.

Figure 6.. Behavioral and neuronal sensitivity to 3D motion

(A) Median psychometric and neurometric curves for each area (MT and FST) and subpopulation (2D and 3D). (B) Distributions of Pearson correlation coefficients comparing simultaneously measured behavioral and neurometric sensitivities for each cue condition (arranged in columns following A).

Figure 7.. Hierarchical model of 3D motion encoding

(A) Model schematic. The first layer included two populations of 2D direction selective units with left- (blue) or right- (green) eye ocular dominance. Pairs of first-layer units with different 2D direction preferences and ODs were linearly combined followed by an expansive non-linearity (inset) in the second layer to create 3D selective units (black). (B) Directions of motion. Directions were sampled across eight azimuths (red) and seven elevations (blue), for a total of 42 unique trajectories. (C) Responses of a second-layer unit that preferred away motion, plotted using an equal-area Lambert projection. Yellow hues indicate higher firing rates and blue hues indicate lower firing rates. The preference for away motion was independent of whether the left eye (left image), the right eye (middle image), or both eyes (right image) were visually stimulated, and therefore the unit was cue invariant. See also Figures S6 and S7.