Segmentation in the perception and memory of events

Abstract

People make sense of continuous streams of observed behavior in part by segmenting them into events. Event segmentation seems to be an ongoing component of everyday perception. Events are segmented simultaneously at multiple timescales, and are grouped hierarchically. Activity in brain regions including the posterior temporal and parietal cortex and lateral frontal cortex increases transiently at event boundaries. The parsing of ongoing activity into events is related to the updating of working memory, to the contents of long-term memory, and to the learning of new procedures. Event segmentation might arise as a side effect of an adaptive mechanism that integrates information over the recent past to improve predictions about the near future.

Figure I

A schematic depiction of how event segmentation emerges from perceptual prediction and the updating of event models. (a) Most of the time, sensory and perceptual processing leads to accurate predictions, guided by event models that maintain a stable representation of the current event. Event models are robust to moment-to-moment fluctuations in the perceptual input. (b) When an unexpected change occurs, prediction error increases and this is detected by error monitoring processes. (c) The error signal is broadcast throughout the brain. The states of event models are reset based on the current sensory and perceptual information available; this transient processing is an event boundary. Prediction error then decreases and the event models settle into a new stable state.

Figure 1

Temporal changes are perceived as event boundaries, and this affects working memory [17]. Participants read narratives containing sentences (marked here in italics) that either indicated a significant interval of time had passed ('an hour later') or not ('a moment later'). These temporal reference sentences often were identified as event boundaries – particularly in the 'hour later' condition. Subsequent experiments assessed memory for objects mentioned before the temporal shift (creek, in this example). Memory was tested in two ways: by measuring the time it took to read an anaphoric reference to the previously mentioned object, or by asking directly whether the participant recognized the word as having been read. Bottom left: Sentences in the 'hour later' condition were read more slowly than the nearly identical 'moment later' versions. Bottom middle: Anaphoric references to previously mentioned objects were read more slowly in the 'hour later' condition. Bottom right: Recognition memory for previously mentioned objects was less accurate in the 'hour later' condition. Such results indicate that working memory is updated at event boundaries. Error bars are standard errors.

Figure 2

The boundaries between musical movements elicited increased activity at a network of brain regions. (a) In Western concert music, symphonies are made up of movements. The sound wave is plotted, and the breaks between movements are illustrated with red lines. In [21], musically untrained participants listened to two 8–10-min segments of symphonies by William Boyce, consisting of movements lasting an average of 1 min and 10 s, while their brain activity was recorded using fMRI. (b) A ventral network including the ventrolateral prefrontal cortex (VLPFC) and posterior temporal cortex (TEMPORAL) increased in activity first at movement boundaries. (c) A dorsal network including the dorsolateral prefrontal cortex (DLPFC) and posterior parietal (PPC) increased in activity slightly later. Both networks were lateralized to the right hemisphere. The authors interpreted these data by suggesting that the response of the ventral network reflected the processing of violations of musical expectancy, whereas the response of the dorsal network reflected consequent top-down modulation of the processing of new musical information. Reproduced, with permission, from [21].