Neural correlates of category learning in monkey inferior temporal cortex

Abstract

We trained two monkeys implanted with multi-electrode arrays to categorize natural images of cats and dogs, in order to observe changes in neural activity related to category learning. We recorded neural activity from area TE, which is required for normal learning of visual categories based on perceptual similarity. Neural activity during a passive viewing task was compared pre- and post-training. After the category training, the accuracy of abstract category decoding improved. Specifically, the proportion of single units with category selectivity increased, and units sustained their category-specific responses for longer. Visual category learning thus appears to enhance category separability in area TE by driving changes in the stimulus selectivity of individual neurons and by recruiting more units to the active network.

Figure 1:

A, schematic of the experimental timeline. Neural activity was compared before and after monkeys learned to categorize natural images of cats and dogs. Monkeys were trained with 40 images and tested using 480 similar held-out images. All passive viewing sessions used all 520 images, randomly interleaved in blocks. B, the 40 images used for training. C, behavioral data from the category training, colored by session number; i, fraction correct trials split by session sextile; ii, same as i but for the first 480 completed trials of the transfer testing session, in which monkeys had only one opportunity to categorize each test image; iii, session averages for the data shown in i and ii; iv, reaction times for correct cat trials (release-on-red trials, see Methods), which correspond approximately to the time it takes monkeys to categorize the image. Note the increased reaction times on the transfer test day, indicating non-expertise.

Figure 2:

A, i Utah array locations in both monkeys, ii number of single units recorded from each pre/post-training session, from each Utah array. B, time-course of abstract category SVM decoders (see Methods), trained on neural population response vectors (spike counts in each 100 ms bin) (mean +/− shaded s.e.m.). Bouts of significant pre- vs. post-training difference were determined with t-tests and a cluster-based permutation procedure that uses trial-shuffled spike counts. C, accuracy of the abstract category SVM decoders in the 175–275 ms bin, across experimental sessions (mean +/− s.e.m.). D, accuracy of abstract category SVM decoders in the 175–275 ms bin, with increasing numbers of the top 100 units used for training (see Methods). Broken vertical lines: half-maximal accuracy for pre- (blue) and post-training (red), respectively. See Supp Fig 3C for sigmoid parameters.

Figure 3:

A, fraction of single units significant in a GLM regressing image category vs. spike count 175–275 ms after image onset. B, all single units’ responsiveness to cats or dogs, across sessions, as measured by the fraction of images from each category that evoked a significant visual response. Black dotted line, unity; red line, best-fit line from major-axis regression. Slopes of fit lines = 1.20 and 1.06, Pearson’s correlations = 0.95 and 0.98. C, distributions of the category selectivity shown in B, summarized for each unit by the absolute difference between the fraction of cat and dog images evoking a significant response. Triangle and line above the histograms represent mean and std, respectively. *, p < 0.05; ***, p < 0.001. D, estimated slope (from major-axis regression; mean + 95% confidence intervals) for best-fit lines in b.

Figure 4:

A, accuracy of abstract category SVM decoders, trained and tested on neural population response vectors from different timepoints. The same set of top 100 units was used for all train/test combinations. B, time-courses of significant difference of category-averaged responses for all units. Each row represents a single unit. Yellow represents significance (see Methods, p < 0.01). C, net proportion of units showing a significant difference at each timepoint in A. Marks above the data represent significant pre-post difference (two-sided chi-square test, p < 0.05). D, probability distribution for the durations of all single unit time courses (not including zero duration) of significant single-unit category coding from the analysis in B (p < 0.05, one-sided rank-sum test).