Extinction reveals that primary sensory cortex predicts reinforcement outcome

Abstract

Primary sensory cortices are traditionally regarded as stimulus analysers. However, studies of associative learning-induced plasticity in the primary auditory cortex (A1) indicate involvement in learning, memory and other cognitive processes. For example, the area of representation of a tone becomes larger for stronger auditory memories and the magnitude of area gain is proportional to the degree that a tone becomes behaviorally important. Here, we used extinction to investigate whether 'behavioral importance' specifically reflects a sound's ability to predict reinforcement (reward or punishment) vs. to predict any significant change in the meaning of a sound. If the former, then extinction should reverse area gains as the signal no longer predicts reinforcement. Rats (n = 11) were trained to bar-press to a signal tone (5.0 kHz) for water-rewards, to induce signal-specific area gains in A1. After subsequent withdrawal of reward, A1 was mapped to determine representational areas. Signal-specific area gains, estimated from a previously established brain-behavior quantitative function, were reversed, supporting the 'reinforcement prediction' hypothesis. Area loss was specific to the signal tone vs. test tones, further indicating that withdrawal of reinforcement, rather than unreinforced tone presentation per se, was responsible for area loss. Importantly, the amount of area loss was correlated with the amount of extinction (r = 0.82, P < 0.01). These findings show that primary sensory cortical representation can encode behavioral importance as a signal's value to predict reinforcement, and that the number of cells tuned to a stimulus can dictate its ability to command behavior.

Fig. 1

Experimental protocols and timeline. There were two behavioral training phases, “Acquisition” and “Extinction”, followed by electrophysiological recording mapping from A1. Acquisition. Rats were trained to bar-press (B) to a 5.0 kHz signal tone (T) for a water reward (R). At least one rewarded bar-press during the tone allowed an additional rewarded bar-press during a “Free” period up to 7 s immediately after the tone ended. The first bar-press after the reward period generated a time-out error-period signaled by a flashing light (E). All other erroneous responses during the silent inter-trial intervals generated an error time-out period 50% of the time. TOTE Test. Rats were tested without Free periods to determine the degree to which they exhibit such a pattern of response in which bar-presses are made beginning at tone onset and continue until the delivery of the first error. This pattern (as shown) reveals use of a learning-strategy called TOTE (“tone-onset-to-error”). The use of TOTE is quantified by a TOTE learning index (TLI) which is used to predict the amount of frequency-specific area gain in A1 (see Fig. 2). Extinction. Typical extinction trials present only a single signal-frequency without reinforcement. Here, we expanded extinction training to include trials that presented non-signal frequencies called “generalization-test” tones. This could reveal the frequency-specificity of extinction by comparison of the decrement in bar-pressing to the signal-tone vs. non-signal probe tone frequencies. Spontaneous recovery. The amount of recovery of bar-press responses to tones was tested 2 days after extinction to assess the strength of extinction. This session was identical to that of extinction training. Electrophysiology. Frequency areas of representation were determined by microelectrode mapping procedures once, at the end of all training. Top right, bar indicates the scale for a 5 s time period in the illustration. Right column references equation indices provided in Methods that used data from the indicated part of the experiment.

Fig. 2

Relationship between the acquisition of a tone-reward association and area of representation of the signal frequency in A1. A significant linear relationship between behavior and brain has been previously established for the tone-reward instrumental task used in this study (equation indicated in figure; [EQ. 1]). The predictive behavior is the degree to which animals solve the task by use of a “tone-onset-to-error” (TOTE) strategy, viz., bar-pressing from onset of a 10 s tone until receiving an error signal (cued by a flashing light during a time-out error-period) for responding in silence after tone offset. The quantification of the degree of use of the TOTE strategy by the TOTE Learning Index (TLI; see Methods, “TOTE Learning Index”) (x-axis), predicts the percent of A1 area representation of the 5.0 kHz signal frequency (within a ±0.25 octave band) (y-axis). The more use of the TOTE learning strategy (i.e., and the higher the TLI), the larger the area in A1. Reproduced from Fig. 7, Bieszczad & Weinberger (2010b).

Fig. 3

Performance and cortical gains in acquisition. (A) Animals learned to bar-press to tones for rewards to a stable level of performance within 17 days of training. Dashed line indicates transition in difficulty from a protocol with short inter-trial-intervals (first four days = 4–12 s, random schedule), to long inter-trial interval durations (5–25 s, random schedule) (Methods, “Acquisition”). (B) Cortical map data from animals trained identically in a prior study (Bieszczad and Weinberger 2010b) were used to estimate the gains in representational area in the current study. The learning curves for the current group and the previous group were the same (inset). Bieszczad and Weinberger (2010b) found a performance correlate (TLI, x-axis) of signal-specific area gain in A1 (y-axis, amount of A1 area for the 5.0 kHz signal frequency ± 0.25 octave) (open circles, n = 7; as in Fig. 2). Here, the established linear relationship was used to estimate the magnitude of signal-specific increase in representational area with the acquisition of the tone–reward association (closed diamonds, n = 11; jiggered on the x-axis dimension to avoid overlap). Table 2 provides all TLI and estimated area values. The size of A1 area representation for the signal frequency band in naïve animals is shown by the horizontal solid line (mean ± standard error) (7.7% ± 1.6). The current group mean area is indicated by the dashed line (12.1% ± 3.2; see also Fig. 6).

Fig. 4

Extinction behavior. Extinction training resulted in a decrease in the number of responses to all tones. A significant group × day interaction (F_4,98 = 5.97, p < 0.001) revealed the decrement in behavioral response was dependent on tone frequency. The greatest decrease in responses was for the signal frequency (solid line, closed circles) and less to frequencies that were more spectrally distant from the signal frequency (Neighbor tones: 2.4 and 10.6 kHz, dashed line, open circles; Far tones: 1.1 and 22.4 kHz, dotted line, open circles).

Fig. 5

Magnitudes of signal-specific response during extinction and spontaneous recovery. The signal (5.0 kHz) and four novel non-signal tone frequencies were presented during extinction and spontaneous recovery sessions. Signal-specific responses in extinction (“SSRE”) values reveal the amount of decrement in signal-specific behavioral responses in extinction (see Methods, “Extinction”). Three examples are illustrated. (a) On the 1^st and 3^rd extinction days, peak responses were at the flanking high frequency while they were at the signal frequency on day 2 and during the test for spontaneous recovery. (b) Peak responses were at the signal frequency throughout extinction training and spontaneous recovery. (c) Peak responses were at the flanking higher frequency on Day 1 of extinction. Although the peak was at the signal frequency on Day 2, the frequency response profile was still broad, as it was to a lesser extent during spontaneous recovery. The range of SSRE values in these examples indicates a high degree of inter-individual variability of extinction learning. Nonetheless, the general pattern was a decrease in SSRE (i.e., decreased responses to the signal-frequency) across extinction training and some degree of spontaneous recovery centered at the signal frequency thereafter. Abbreviations: Ext D1, Ext D2, Ext D3 = Days 1, 2 and 3 of extinction training, respectively; Spon. Rec = spontaneous recovery test session; SSRE = Signal-specific responses in extinction (see text; [EQ. 4]). SSRE values for all animals during all extinction and spontaneous sessions are provided in Table 1.

Fig. 6

Representation of frequency in A1 following all training. (A) The amount of characteristic frequency (CF) representational area for the signal frequency after extinction was determined relative to the total size of A1 (y-axis, % of total). CF distributions in half-octave bands (centered at the 5.0 kHz signal frequency) revealed a significant difference in the interpolated A1 representation for the signal frequency (5.0 kHz ± 0.25 octaves) from 7.7% (±1.6 s.e.m.) in naives to 12.5% (±3.4 s.e.m.) in trained subjects after acquisition (filled circle; t₁₈ = 3.58, *p < 0.001). However, extinction reduced the signal area from 12.1% ± 3.2 to 7.5% (±3.7 s.e.m.) (filled symbols). In contrast, the areas of non-signal frequencies were not significantly reduced (all F_1,20 ratios, p > 0.05). (B) A1 maps of trained subjects that varied in the amount of cortical representation of the signal frequency. Examples show that subjects could have signal areas that were not significantly different in size from naïve (e.g., 7.7%), or with significantly more area (e.g., 10.6%) or less signal area (e.g., 2.2%) than naïve. Thus, the effects of extinction on the representation of frequency in A1 could differ between subjects.

Fig. 7

Behavioral extinction is significantly related to the magnitude of area reduction in A1. There was a significant relationship between the estimated area of change (mainly loss) of signal-specific representational area in A1 and the strength of behavioral extinction: the greater the reduction in signal area, the stronger the extinction. Best fit regression was curvilinear: y = −1.37x² + 35.17x − 121.94 (r = 0.82, p < 0.01; n = 11). The x-axis shows the magnitude of reversal in the signal-specific area estimated to have been gained in A1 for the signal-frequency (in a half-octave band centered at the signal-frequency). Larger positive values along the x-axis indicate greater reductions in signal area with extinction. Larger negative values indicate lesser reductions. The magnitudes of individual extinction-induced area loss were calculated as the difference between the estimated gain in area due to acquisition (Fig. 3B) and the signal-specific area determined electrophysiologically after extinction (Fig. 6). The y-axis shows the strength of extinction for the signal tone (5.0 kHz) as the percentage of the total decrement of behavioral responses during the three days of extinction that recovered two days later during the spontaneous recovery test. The significant relationship between signal area and memory strength is curvilinear likely because the amount of possible loss in A1 area reaches a floor because the amount of signal-specific representational area cannot decrease to a negative value. Annotated diamonds (a, b and c) reference individual animals shown in Fig. 5A, 5B and 5C in extinction and spontaneous recovery, respectively.