Input-Specific Gain Modulation by Local Sensory Context Shapes Cortical and Thalamic Responses to Complex Sounds

Abstract

Sensory neurons are customarily characterized by one or more linearly weighted receptive fields describing sensitivity in sensory space and time. We show that in auditory cortical and thalamic neurons, the weight of each receptive field element depends on the pattern of sound falling within a local neighborhood surrounding it in time and frequency. Accounting for this change in effective receptive field with spectrotemporal context improves predictions of both cortical and thalamic responses to stationary complex sounds. Although context dependence varies among neurons and across brain areas, there are strong shared qualitative characteristics. In a spectrotemporally rich soundscape, sound elements modulate neuronal responsiveness more effectively when they coincide with sounds at other frequencies, and less effectively when they are preceded by sounds at similar frequencies. This local-context-driven lability in the representation of complex sounds-a modulation of "input-specific gain" rather than "output gain"-may be a widespread motif in sensory processing.

Figure 1

Local Context Shapes Input-Specific Gain (A–D) Cartoon illustrations of receptive field integration mechanisms. (A) In the most basic scheme, input stimuli (gray-level spectrogram) are integrated by a single set of fixed weights (orange). Pointwise nonlinear transforms may apply to each specific input (not shown) or to the integrated weights (light green). (B) Multidimensional LNP models include a small number of differently weighted overlapping integration fields, with outputs combined by a multi-imensional nonlinearity (green). Methods such as MID and STC are designed to characterize such models. (C) Normalization, or other variable global gain, involves the output of one field (blue) modulating the gain of the integrated response to the other (orange). The normalization field may extend well beyond the integration field, so that the effective gain reflects global statistical properties of the stimulus. A further nonlinear transformation (light green) may act before or after gain modulation (light blue). (D) In the phenomenon described here, local context (blue) around each input shapes the gain of response to that specific input. Each input experiences a different context and thus a potentially different gain. Gain-modulated inputs are integrated (green), with a possible further nonlinear transformation (light green). (E) The CGF model. The contextual input-specific gain model incorporates two sets of time-freqency weights. The Principal Receptive Field (PRF; ${\mathbf{w}}^{\mathbf{t}\mathbf{f}}$) describes the basic sensitivity of the neuron to spectrotemporal energy at all frequencies within a short time window, analogous to the STRF. The Contextual Gain Field (CGF; ${\mathbf{w}}^{\mathit{\tau }\varphi }$) describes how each sensitivity is modified by its local acoustic context. The model can be viewed as acting in two stages. First, the stimulus spectrogram is convolved with the CGF in both time and frequency to estimate the local input-specific gain at each spectrotemporal point (upper green arrow). The local stimulus power is then scaled by the corresponding gain and these scaled values, weighted by the PRF, are summed to model the neural response (lower green arrow). The measured response (peri-stimulus-time histogram or PSTH) for one example cortical neuron is shown (gray bars) along with the rates predicted by the CGF model (bright green) and an unmodified STRF (dull gray-green). Differences in prediction (black triangles) show that local contextual gain effects both increase and decrease firing rates relative to the STRF model of static sensitivities. (F–J) Local input-specificity of contextual gain effects. The relationships between the measured responses of one example unit and the sound level at two spectrotemporal locations within the unit’s PRF far enough apart in time and frequency to be subject to different local sound contexts (F) are shown without reference to local context (gray open circles and dashed lines); sorted by whether the integrated contextual energy in a local window around that spectrotemporal location fell within a low, middle or high quantile (G and H, colored circles and lines); or, as a control, sorted according to “distant” contextual energy — i.e., integrated energy around the other of the two input locations (I and J, colored circles and lines). Error bars indicate standard error in the mean; lines are fit to the empirical data. The slopes of the input-response relationships differ when sorted by local spectrotemporal context (black bars with asterisks indicate significance), but not when sorted by contextual energy at the spectrotemporally distant location.

Figure 2

Contextual Input-Specific Gain Shapes Both Cortical and Thalamic Responses (A) Scatterplot of generalization performance for the CGF and STRF models in cortex and thalamus measured by cross-validation; inset shows histogram of differences in favor of the CGF model (left) or STRF model (right). Black dashed lines indicate equal performance. The CGF model almost always generalizes more accurately than the STRF, showing that contextual input-specific gain plays a substantial role in shaping responses in both brain structures. (B and C) Predictive power extrapolations for CGF model (bright colors) and STRF model (dull, greyed colors) in cortex (B) and thalamus (C). Filled circles and solid lines indicate generalization performance on test data, assessed by cross-validation; open circles and dashed lines show predictive performance on training data. In the zero-noise limit, extrapolated intercepts (indicated on the left) are all higher for the CGF model. See Supplemental Experimental Procedures for further explanation. (D) Effective input-specific gains and predictive advantage. Each dot and horizontal bar indicates the median and interquartile range of the distribution of effective input-specific gains across all points in the stimulus for one neuron, obtained by convolving the spectrogram of the DRC stimulus with the neuron’s CGF (see also Figure 3). Median input-specific gains tend to be substantially smaller than 1 and interquartile ranges are often large, indicating that effects of local acoustic context are predominantly suppressive but can vary substantially across spectrotemporal points within the DRC stimulus.

Figure 3

Variation in Contextual Input-Specific Gain across Spectrotemporal Points within a Complex Stimulus (A) Two-second-long segment of the DRC stimulus. (B–E) CGFs (left) for four example cortical neurons are convolved with the spectrogram of the DRC stimulus to reveal effective input-specific gains (right) that vary substantially from cell to cell, frequency to frequency and moment to moment within the stimulus.

Figure 4

Structure of Input-Specific Gain Modulation in the Cortex and Thalamus (A) Example CGF and PRF pairs for four neural recordings in cortex (left) and four recordings in thalamus (right). CGFs (top) range over relative time $\mathit{\tau }$ and relative frequency $\varphi$. Weights represent the change in gain induced if one of the loudest tones of the DRC stimulus were to fall at the corresponding location, and are shown on common scale (left). PRFs (bottom) range over time $\mathit{t}$ prior to the modeled response and acoustic frequency $\mathit{f}$ (log-spaced). Stimulus modulation of firing differs substantially across neurons, so PRFs are separately (and symmetrically) scaled to the maximum change in firing rate shown above each one. (B–E) Mean CGFs and average profiles in cortex (green) and thalamus (magenta). The central panel (C) shows the spectrotemporal pattern of the mean CGF weights in both structures. The average spectral profiles (B), spectral profiles at 0 delay (D) and average temporal profiles (E) of both means are shown superimposed, with error bars indicating the SE of the estimated population means.

Figure 5

Generalization Disadvantage for Models with Key Features of CGF Elided (A–C) Cortex; (D–F) thalamus. Each panel contrasts the effects of eliding parameters in two identically sized sections of the CGF (gray rectangles): one corresponding to a CGF feature that appeared to consistently shape input-specific gain, the other a control section where CGF weights were inconsistent or small. Weights in the elided regions were fixed at zero, and the model was re-fit to optimize the remaining model parameters. Histograms show distribution across neurons of differences in cross-validation predictive performance (generalization accuracy) relative to the unelided CGF model; p value indicates significance threshold at which the hypothesis that median change in performance equals or exceeds zero can be rejected (one-tailed sign test, uncorrected; N = 64 in cortex and 101 in thalamus). Scatter plots compare generalization accuracy of the two elided models neuron-by-neuron; p value indicates threshold for rejection of the hypothesis that median difference for feature elision minus control elision equals or exceeds zero (one-tailed sign test, uncorrected). Across the neural population, elision of key CGF features always resulted in poorer generalization accuracy than that achieved by the full (unelided) model. By contrast, control elisions had significantly less impact; the hypothesis that control elisions produced no reduction in predictive performance could not be rejected in any case after correction for multiple comparisons.

Figure 6

Variability in CGF Structure across Neurons (A–C) PCA of CGFs in the cortex. (A) The absolute variance (i.e., average squared Δ gain) captured by each of the first 32 PCs. (PC numbers are plotted logarithmically.) (B) Filled symbols: fractional variance in the CGFs captured by the leading PCs, as a function of number of PCs considered. Open symbols: fractional sum of squares of the mean cortical CGF that projects into the space spanned by the leading PCs, demonstrating how well the variance is aligned with the mean. (C) The three leading PCs in order from left to right. (D–F) PCA of CGFs in the thalamus. Subpanels correspond to (A)–(C).

Figure 7

Contextual Input-Specific Gain Compared between Two Cortical Fields and Two Thalamic Subdivisions (A and B) Mean CGFs of neurons in cortical areas A1 and AAF. Overall structure is similar in both areas, but the delayed suppression region is shifted toward shorter delays in AAF. (C) Mean temporal CGF profiles averaged over frequency offset for both areas (error bars show standard errors in the mean). The shorter delay and shorter duration of the suppressive contextual gain effect within AAF is clearly evident. (D) Mean spectral CGF profiles at zero time lag (error bars show standard errors in the mean). The general shape of the spectral interaction is similar in the two cortical areas, although side peaks in AAF fall at slightly larger frequency offsets, perhaps as a result of the stronger short-delay suppression in AAF. (E–H) Similar figures show contextual gain effects in the ventral and medial subdivisions of MGB. No substantial differences are observed between these two thalamic subdivisions.