Tracking neural representations of attended and unattended features in multisensory working memory over time

Abstract

Working memory supports goal-directed behavior by maintaining task-relevant information. However, a growing number of studies shows that task-irrelevant features can interfere with the recall of task-relevant information. While this phenomenon is well documented in unisensory contexts, it remains unclear whether and how task-irrelevant information persists in multisensory working memory. Here, we examine the role of cross-modal binding by tracking the dynamic neural representation of audio-visual objects under varying selective attention conditions, using an EEG-based audio-visual delayed-match-to-sample task. Participants attended to either auditory or visual features (selective attention), or to both features (conjunction) of two sequential audio-visual items and subsequently compared those task-relevant features to an audio-visual probe. Further, to investigate the influence of bottom-up factors on cross-modal binding, we manipulated spatial congruency by presenting both features from either the same or from disparate positions. Behaviorally, task-irrelevant features interfered with performance even under selective attention, consistent with automatic cross-modal binding and encoding into working memory. Condition-level representational similarity analysis (RSA) showed that EEG activity patterns under selective attention more closely resembled those of conjunction trials than unisensory trials, indicating that task-irrelevant features were incorporated into multisensory object-level representations. This conjunction-similarity persisted in attend-visual trials, but declined over time in attend-auditory trials, reflecting partial filtering of task-irrelevant orientations. Crucially, activity patterns never shifted fully towards an auditory-only profile, indicating that irrelevant visual features were not fully excluded. Overall, these results demonstrate the persistence of task-irrelevant information in multisensory working memory and offer critical insights into how attentional processes shape its representational architecture.

Keywords: Cross-modal binding; Electroencephalography (EEG); Multisensory processing; Representational similarity analysis (RSA); Selective attention; Working memory.