Individual differences reveal limited mixed-category effects during a visual working memory task

Abstract

Using stimuli from different categories may expand the capacity limits of working memory (WM) by spreading item representations across distinct neural populations. We explored this mixed-category benefit by correlating individuals' behavioral performance with fMRI measures of category information during uniform- and mixed-category trials. Behaviorally, we found weak evidence for a mixed-category benefit at the group-level, although there was a high degree of individual variability. To test whether distinct neural patterns elicited superior performance in some individuals, we correlated a multivariate measure of neural category information with multiple behavioral metrics. This revealed a widespread positive relationship, intuitive for hit rate and working memory capacity, but counterintuitive for false alarm rate. Overall, these data suggest that mixed-category effects may support working memory performance, but unexpectedly, not all participants show this benefit. Only some people may be able to take advantage of representing mixed-category information in a differentiable way.

Keywords: Multivariate pattern analysis; Object category; Working memory.

Figure 1:

Trial sequence for the visual WM task. Participants were instructed to maintain fixation on the central square. Participants’ task was to indicate with a button press if any of the objects presented during the retrieval phase differed from those presented during the encoding phase. The example trial shows a mixed-category trial in which an object change occurred. On uniform-category trials (not shown), all four objects in the encoding and retrieval periods were selected from the same object category. On catch trials (not shown), the trial ended after encoding, excluding maintenance and retrieval phases and the color of the inter-trial interval fixation spot was red. In the experiment, the display for each phase was the same size; here, the encoding and retrieval phases are enlarged for visibility.

Figure 2:

Behavioral data. Solid black circles represent the mean difference between mixed-category and uniform-category trials (mix-unif) for each behavioral metric (HR = hit rate; FA = false alarm rate; K = working memory capacity). Black lines represent the 95% CI. Light gray circles represent each of the 24 participants, with random horizontal jitter to facilitate visualization.

Figure 3:

Multivariate analyses demonstrate robust category representations during the WM task. Group-level searchlight analysis of category information (Δz) across participants during encoding (A), maintenance (B), and retrieval (C). This analysis was restricted to uniform-category trials, in which stimuli from a single category (body or objects) was present. Hot colors represent regions in which category information was significantly greater than zero (p < .05, uncorrected), indicating that body and object stimuli evoked consistent patterns of BOLD activity in these regions. A selection of major sulci for these views are depicted in Figure 4.

Figure 4:

Behavioral-neural correlations reveal that greater separability in neural patterns for objects and bodies was generally positively correlated with individual differences in false alarm rates across mixed and uniform-category trials. This was unexpected, as it suggests that more distinct neural patterns across categories were associated with worse behavioral performance on mixed-category trials. Behavioral-neural correlations (r_behav:neural) were calculated between category information (Δz) and FA_mix-unif separately for encoding (A), maintenance (B), and retrieval (C). The clusters represent regions in which the conjunction of r_behav:neural (p < .05, uncorrected) and category information (Δz; p < .05, uncorrected; see Figure 3), were robust (p < .0025, surface-based cluster correction to family-wise error α = 0.05). Hot colors represent significant positive r_behav:neural values. Several major sulci are labeled for reference: calcarine sulcus (calc), cingulate sulcus (cing), collateral sulcus (cos), central sulcus (cs), intraparietal sulcus (ips), postcentral sulcus (pcs), parietooccipital sulcus (pos), superior temporal sulcus (sts).

Figure 5:

Behavioral-neural correlations reveal that greater separability in neural patterns for objects and bodies was generally positively correlated with individual differences in hit rates across mixed and uniform-category trials. This is consistent with the hypothesis that more distinct neural patterns across categories are associated with better behavioral performance on mixed-category trials. Behavioral-neural correlations (r_behav:neural) were calculated between category information (Δz) and HR_mix-unif, separately for each phase of the trial: encoding (A), maintenance (B), and retrieval (C). Conventions and abbreviations are the same as in Figure 4.

Figure 6:

Behavioral-neural correlations reveal that greater separability in neural patterns for objects and bodies was generally positively correlated with individual differences in working memory capacity across mixed and uniform-category trials. This is consistent with the hypothesis that more distinct neural patterns across categories are associated with better behavioral performance on mixed-category trials. Behavioral-neural correlations (r_behav:neural) were calculated between category information (Δz) and K_mix-unif, separately for each of the three phases of the visual WM task: encoding (A), maintenance (B), and retrieval (C). Conventions and abbreviations are the same as in Figure 4.