Neural computations governing spatiotemporal pooling of visual motion signals in humans

Abstract

The brain estimates visual motion by decoding the responses of populations of neurons. Extracting unbiased motion estimates from early visual cortical neurons is challenging because each neuron contributes an ambiguous (local) representation of the visual environment and inherently variable neural response. To mitigate these sources of noise, the brain can pool across large populations of neurons, pool the response of each neuron over time, or a combination of the two. Recent psychophysical and physiological work points to a flexible motion pooling system that arrives at different computational solutions over time and for different stimuli. Here we ask whether a single, likelihood-based computation can accommodate the flexible nature of spatiotemporal motion pooling in humans. We examined the contribution of different computations to human observers' performance on two global visual motion discriminations tasks, one requiring the combination of motion directions over time and another requiring their combination in different relative proportions over space and time. Observers' perceived direction of global motion was accurately predicted by a vector average readout of direction signals accumulated over time and a maximum likelihood readout of direction signals combined over space, consistent with the notion of a flexible motion pooling system that uses different computations over space and time. Additional simulations of observers' performance with a population decoding model revealed a more parsimonious solution: flexible spatiotemporal pooling could be accommodated by a single computation that optimally pools motion signals across a population of neurons that accumulate local motion signals on their receptive fields at a fixed rate over time.

Figure 1.

Examples of RDKs used in the temporal and spatiotemporal experiments. In each experiment, observers judged whether sequentially presented standard or comparison RDKs had a more clockwise direction of motion. Temporal and spatial dot directions were sampled with replacement from an asymmetrical probability distribution with distinct measures of central tendency. All dots in the temporal comparison were displaced in the same randomly sampled direction on each image, generating a temporal sequence of directions across images; individual dots in the spatial comparison were displaced in independently sampled directions on each image, generating a spatial distribution of directions on each image. The comparison RDKs used for the spatiotemporal experiments consisted of different mixtures of spatial and temporal dot directions.

Figure 2.

A–C, Distributions of dot directions for different comparison RDKs. Arrows show the median (white), vector average (gray), and modal (black) direction of the comparison RDKs. D, Directions sampled with replacement from the comparison distributions shown in the top and bottom panels of A. When the comparison distribution is symmetrical, the perceived direction of the standard RDKs aligns with the three measures of central tendency of the comparison RDK. When the comparison distribution is asymmetrical, only the vector average direction aligns with the perceived direction of the standard RDK. The smooth lines through the data points are the best-fitting solutions to Equation 1.

Figure 3.

Vector average of direction distributions accumulated over time predicts the temporal pooling of local motion directions. A–C, Symbols show the perceived direction of four observers as a function of different comparison distributions. Lines show the temporal vector average (solid), median (dashed), and modal (dotted) direction of the comparison distribution. B, Note that the median direction has been offset from the mode to reduce clutter. D–F, Symbols show the direction discrimination thresholds of four observers as a function of different comparison distributions. Error bars are 95% confidence intervals (CIs).

Figure 4.

Vector average readout from a population coding model predicts the temporal pooling of local motion directions. A–C, White circles show the average perceived direction of four observers as a function of different comparison distributions. Solid lines show the perceived directions estimated by a vector average decoder (Eq. 6). D–F, White circles show the average direction discrimination thresholds of four observers as a function of different comparison distributions. Black circles show the direction discrimination thresholds estimated by a vector average decoder. Error bars are 95% CIs.

Figure 5.

Spatial and temporal dot directions sampled in different proportions from a single asymmetrical uniform distribution. Rows show different relative percentages of spatial and temporal dot directions. Columns show the samples obtained over time on each positional update of the dots.

Figure 6.

Different population decoders predict the spatial and temporal pooling of local motion directions. A–C, Symbols show the perceived direction of three observers at three stimulus durations, plotted as a function of the percentage of temporal dots (inversely related to the percentage of spatial dots) in the comparison. D, Symbols show the average perceived direction of observers plotted and notated as in A–C. Dashed lines on the right are the perceived direction at three stimulus durations estimated by a vector average (V. Average) decoder (Eq. 6) when all the dots are temporal; black dashed lines on the left of the plot are the perceived direction at three stimulus durations estimated by a maximum likelihood (M. Likelihood) decoder (Eq. 5) when all of dots are spatial (the maximum likelihood estimates are the same for the three durations). Error bars are 95% CIs.

Figure 7.

Direction discrimination thresholds in the spatiotemporal pooling experiment. A–C, Symbols show the direction discrimination thresholds of three observers at three stimulus durations, plotted as a function of the percentage of temporal dots in the comparison. D, Symbols show the average direction discrimination thresholds of observers plotted and notated as in A–C. Error bars are 95% CIs.

Figure 8.

Effects of manipulating the behavior of input neurons on the estimate of the maximum likelihood perceived direction in the spatiotemporal pooling experiment. Left column shows basic model population responses to equally balanced (50%, 50%) spatial and temporal samples from a comparison distribution (inset in each panel), presented for a total duration of 104 ms. The left column shows examples of changes to the shape of the population response caused by varying the following: A, numbers of neurons in the population (N = 180 neurons); C, level at which the responses of the neurons saturate (Eq. 7, R_sat = 40 spikes/s); E, correlation structure of the interal noise across the population of neurons (Eq. 10, C_max = 0.5); G, time constant of temporal response integration (Eq. 8, τ_r = 20 ms); and I, rate at which neurons respond to the number of dot directions on their receptive fields [Eq. 9, Σ(t,D) = 18]. Right column shows how varying the model parameters N, R_sat, C_max, τ_r, and Σ(t,D) modulated the estimate of the maximum likelihood of perceived direction in the spatiotemporal experiment. Varying the rate at which neurons respond to the total number of dots on their receptive field was the only manipulation to the basic model that approximated observers' performance at different stimulus durations.

Figure 9.

Computations governing spatiotemporal pooling of local motion directions. A, Maximum likelihood (M. likelihood) readout from model neurons that sum local directions at a fixed rate over time (Eq. 9, τ_D = 0.36) accurately predicts human observers' perceived direction at different stimulus durations in the spatiotemporal pooling experiment. C, E, Corresponding estimates of perceived direction from winner-takes-all and vector average decoders are accurate but hugely variable (C) and completely inaccurate (E), respectively. B, D, F, Maximum likelihood (B), winner-takes-all (D; W-T-A), and vector average (F; V. average) decoders all approximate the general pattern of observers' direction discrimination thresholds in the spatiotemporal pooling experiment. Error bars are 95% CIs.