Action sharpens sensory representations of expected outcomes

Abstract

When we produce actions we predict their likely consequences. Dominant models of action control suggest that these predictions are used to 'cancel' perceptual processing of expected outcomes. However, normative Bayesian models of sensory cognition developed outside of action propose that rather than being cancelled, expected sensory signals are represented with greater fidelity (sharpened). Here, we distinguished between these models in an fMRI experiment where participants executed hand actions (index vs little finger movement) while observing movements of an avatar hand. Consistent with the sharpening account, visual representations of hand movements (index vs little finger) could be read out more accurately when they were congruent with action and these decoding enhancements were accompanied by suppressed activity in voxels tuned away from, not towards, the expected stimulus. Therefore, inconsistent with dominant action control models, these data show that sensorimotor prediction sharpens expected sensory representations, facilitating veridical perception of action outcomes.

Fig. 1

A schematic illustration of how predictive signals influence populations of sensory neurons under ‘cancellation’ and ‘sharpening’ models. Left: Cancellation models developed in the action control literature propose that when we move (e.g., abduct our index finger) we generate a predictive signal that suppresses activity in sensory units tuned to the expected action outcome (e.g., observation of an index finger movement)^, . Weakening the activity in these units reduces the signal-to-noise ratio of the population response when stimuli are expected, leaving less information in patterns of neural activity that could be detected by a pattern classifier. Right: In contrast, sharpening models of prediction found in the wider sensory cognition literature suggest that predictive signals suppress activity in units tuned away from expected stimuli^, . This leads to a sharper population response when stimuli are expected, with a higher signal-to-noise ratio and making it easier to decode stimulus identity on these trials

Fig. 2

Action execution and observation task, with an illustration of pattern classification. a During the experimental task participants performed actions (e.g., index finger movement), which triggered movements of the avatar hand that were congruent (index finger) or incongruent (little finger) with their actions and in synchrony with them. b A searchlight approach was used to decode the identity of observed actions based on BOLD activity from occipital and temporal regions (shaded in green)

Fig. 3

Stimulus decoding accuracies across regions of interest in the visual brain. Accuracies are shown for ROIs in bilateral occipital cortex (red), left occipital cortex (blue) and right occipitotemporal cortex (green). Decoding accuracy is higher on congruent (saturated) relative to incongruent (desaturated) trials, suggesting higher fidelity sensory representations when stimuli reflect expected action outcomes. Error bars show 95% within-participant confidence intervals of the mean difference between conditions (N=20, 95% $\mathrm{CI∕}\sqrt{2}$)

Fig. 4

BOLD activity across regions of interest in the visual brain. Activity is lower for congruent (saturated) relative to incongruent (unsaturated) trials only in voxels that are tuned away from the observed stimulus (e.g., those voxels tuned towards index fingers on trials where little finger movements are presented). This pattern suggests that the lower BOLD signal found throughout the action literature when perceiving expected action outcomes may reflect a sharpening of expected sensory representations, rather than cancellation. Error bars show 95% within-participant confidence intervals of the mean difference between conditions (N = 20, 95% $\mathrm{CI∕}\sqrt{2}$)