A neural mechanism for optic flow parsing in macaque visual cortex

Abstract

For the brain to compute object motion in the world during self-motion, it must discount the global patterns of image motion (optic flow) caused by self-motion. Optic flow parsing is a proposed visual mechanism for computing object motion in the world, and studies in both humans and monkeys have demonstrated perceptual biases consistent with the operation of a flow-parsing mechanism. However, the neural basis of flow parsing remains unknown. We demonstrate, at both the individual unit and population levels, that neural activity in macaque middle temporal (MT) area is biased by peripheral optic flow in a manner that can at least partially account for perceptual biases induced by flow parsing. These effects cannot be explained by conventional surround suppression mechanisms or choice-related activity and have substantial neural latency. Together, our findings establish the first neural basis for the computation of scene-relative object motion based on flow parsing.

Keywords: motion; neurophysiology; optic flow; perceptual bias; self-motion; visual cortex.

Figure 1.. Illustration of expected perceptual biases from flow parsing and a potential neural correlate.

(A) Schematic illustration of a stimulus condition presenting forward self-motion (green dots) and upward object motion on the screen (yellow dots). Right: If flow parsing occurs, the rightward flow vector at the location of the object (solid green arrow) would be subtracted, leading to a leftward bias in perceived object direction (black arrow). The yellow arrow indicates object direction in image (screen) coordinates. The dashed green arrow is the opposite of the flow vector, which is vectorially added to the image motion (yellow arrow) to obtain the expected perceived direction (black arrow). (B) Same as panel A except that optic flow simulates backward self-motion (red dots), leading to a rightward expected bias from flow parsing. (C) Hypothetical neural population response profiles in response to a presentation of an object (stimulus applied to the receptive fields of neurons) in the right visual hemi-field moving straight upward (0 deg, yellow arrow) in retinal coordinates (e.g., yellow dots in panels A and B). Each curve shows an idealized neural population response, in which the normalized average response of each idealized neuron is plotted as a function of its preferred direction relative to vertical. The response of each neuron was normalized such that its maximal response was assigned a value of 1.0. When an observer is stationary (blue), the population hill of activity peaks at 0 deg (vertical motion, yellow arrow), such that an estimate of direction from the population activity would be unbiased. If the perceptual biases induced by flow parsing are reflected in this neural population response, then forward self-motion should shift the curve leftward (green) and backward self-motion should shift the curve rightward (red). As a result, a neuron that prefers a direction of −45° should have a greater response during forward self-motion than during backward self-motion. In contrast, a cell that prefers +45° would show the opposite effect. Crucially, note that image motion of the object on the screen is identical across self-motion conditions, such that the curves should be superimposed if there is no neural correlate of flow parsing. See also Video S1.

Figure 2.. Optic flow systematically biases object motion perception in monkeys.

(A) Psychometric functions from a recording session in which monkey P discriminated object direction in the presence of optic flow. Symbol shape and color denote data from the stationary (blue squares), forward (green circles), and backward (red triangles) self-motion conditions. Smooth curves show fits of a cumulative Gaussian function to the data points. Horizontal error bars indicate 95% confidence intervals on the PSEs, and the dashed vertical lines indicate the expected PSEs for complete flow-parsing (FP gain = 1). Because the object was presented in the right visual hemi-field, monkey P’s perception of object motion was biased leftward during forward self-motion and rightward during backward self-motion. (B) Data from a recording session in which monkey M discriminated object motion in the left hemi-field, leading to an opposite pattern of perceptual biases. (C) Distributions of flow-parsing gains (observed/expected PSE shift) for 20 recording sessions from monkey M (teal) and 19 recording sessions from monkey P (purple). Downward-pointing triangles indicate the median flow-parsing gains for each animal. See also Figures S1, S5, and S6.

Figure 3.. Modulation of MT firing rates by optic flow depends on direction tuning.

(A-C) Firing rates of three MT units, two of which (A,B) were recorded during the same session in which the object was in the left visual hemi-field. Data are shown separately for the stationary (blue squares), forward (green circles), and backward (red triangles) self-motion conditions. Error bars denote SEM. (A) For a unit that prefers leftward object motion, responses during backward self-motion are greater than responses during forward self-motion. (B) For a simultaneously recorded unit that prefers rightward object motion, responses during forward self-motion are greater than those during backward self-motion. (C) For a unit that prefers vertical motion, there is little difference in response between the self-motion conditions. (D-E) Flow-modulation index (FMI) across the population of 727 units depends systematically on aspects of direction tuning. Color and shape of symbols denote monkey identity: monkey M (teal triangles) and monkey P (purple circles). (D) FMI is circularly correlated with preferred direction, where a preference of 0 denotes the upward task direction reference. The black trace denotes a running mean FMI when data are pooled across monkeys; purple and teal traces denote the running means separately for monkeys P and M, respectively. (E) FMI is inversely correlated with selectivity for horizontal motion, as measured by the horizontal direction discrimination index (HDDI). Black line indicates the line of best fit (linear regression) when data are pooled across monkeys; purple and teal lines denote linear fits for monkeys P and M, respectively. See also Figures S2 and S4.

Figure 4.. Effects of optic flow on MT responses are distinct from choice-related activity.

Neural responses were analyzed to dissociate effects of background optic flow (Flow Probability, FP) from choice-related response modulations (Choice Probability, CP), as detailed in Methods. (A, B) Flow probability is robustly correlated with both neuronal preferred direction and HDDI, similar to the results for FMI (format as in Figure 3D,E). (C, D) In contrast, choice probability is not systematically related to either flow probability or HDDI. This reveals that effects of object flow background on MT responses are dissociable from choice-related modulations. See also Figures S3 and S4.

Figure 5.. Time course of population response, HDDI, and FMI.

In each panel, vertical lines indicate stimulus onset and offset, and the gray curve indicates the stimulus velocity profile. (A) Time course of normalized response to vertical (0 deg) object motion, averaged over all units. Color indicates self-motion direction; blue: stationary, green: forward, red: backward. (B) Mean HDDI time course for subsets of units, grouped according to the horizontal component of their preferred direction. Darker red curves indicate units with preferred directions closer to leftward (−90 deg), and darker blue curves indicate units with preferred directions closer to rightward (+90 deg). Horizontal black lines indicate time periods during which HDDI differs significantly between the darkest red and the darkest blue curves. (C) Mean FMI time course for subsets of units, grouped according to their HDDI values, where negative/positive HDDI values indicated a preference for leftward/rightward motion. Horizontal black lines indicate time periods during which FMI differs significantly between the darkest red and the darkest blue curves.

Figure 6.. Psychometric functions representing monkey behavior and population decoder performance for two example sessions.

Format as in Figure 2A,B. Vertical lines indicate the expected PSEs for complete flow-parsing. (A) Psychometric function reflecting one session of monkey P’s direction discrimination performance for an object located in the left visual hemifield. (B) Predicted psychometric function produced by the stimulus_world decoder, which was trained to discriminate object direction in world-centered coordinates from neural responses in the same for which the behavioral data are shown in panel A. (C) Psychometric function produced by the stimulus_screen decoder, which was trained to discriminate object direction in retinal coordinates (same session as panels A, B). (D-F) Psychometric data and decoder performance for one example session from monkey M.

Figure 7.. Summary of comparison between monkey behavior and population decoder performance.

Each datum represents one experimental session from monkey M (teal triangles) or monkey P (purple circles). Star-shaped symbols indicate the median perceptual and decoder FP gains across sessions for each animal. (A) FP gains of the stimulus_world decoder are plotted against the monkeys’ perceptual FP gains. (B) FP gains of the stimulus_screen decoder are plotted against the monkeys’ perceptual FP gains. (C) FP gains of the choice decoder plotted against the monkeys’ perceptual FP gains.