The cognitive neuroscience of visual short-term memory

Abstract

Our understanding of the neural bases of visual short-term memory (STM), the ability to mentally retain information over short periods of time, is being reshaped by two important developments: the application of methods from statistical machine learning, often a variant of multivariate pattern analysis (MVPA), to functional magnetic resonance imaging (fMRI) and electroencephalographic (EEG) data sets; and advances in our understanding of the physiology and functions of neuronal oscillations. One consequence is that many commonly observed physiological "signatures" that have previously been interpreted as directly related to the retention of information in visual STM may require reinterpretation as more general, state-related changes that can accompany cognitive-task performance. Another is important refinements of theoretical models of visual STM.

Figure 1

Dissociating elevated delay-period signal from the short-term retention of information. Summary of results from [11], in which subjects were scanned with fMRI while viewing one, two, or three sample displays of moving dots, then probed to recall the direction of one. A. Univariate statistical maps indicating regions showing load sensitivity during sample presentation, the delay period, or both. B. Time series data from “sample-only” voxels (panel on left) or “delay-only” voxels. Teal waveform illustrates decoding performance of a classifier trained at the time point with the maximal stimulus-evoked response (indicated with dot) then swept across the remainder of the trial. Maroon and solid gray waveforms are the analogous performance of classifiers trained at a time point late in the delay period, or 2 sec prior to sample onset, respectively. Asterisks indicate better-than-chance decoding at p < .05(*) and p < .01(**). Superimposed is the trial-averaged BOLD activity, depicted in the dotted waveform and aligned with the vertical axis on the right-hand side of the plot. C. Plots of neural precision against behavioral precision. Each color corresponds to an individual subject and each digit (3, 2, or 1) to that individual’s neural and behavioral precision at the corresponding memory load. Lines are the fit indicated by ANCOVA (r² = .35).

Figure 2

Neural evidence for AMIs vs. UMIs vs. absent items, on trials when the second retrocue cues the same item as had the first (“Repeat”), or the previously uncued item (“Switch”). Legend labels “cued” and “uncued” refer to an item’s status relative to the first cue. A. MVPA of fMRI data from [47]. Circles along timeline denote sample presentation, triangles denote retrocues, and squares denote recognition probes. Circles at top of plots indicate statistical significance of a stimulus category vs. the empirical baseline of MVPA evidence for the irrelevant category. MVPA classifiers were trained on data acquired in a prior training session. B. MVPA of EEG data from [48]. Graphical conventions are the same as in A, with the exception that statistical significance (only tested during delay periods) is denoted with color-coded asterisks. MVPA classifiers were trained and tested on the same dataset using hold-one-trial-out cross validation.