Temporal binding as multisensory integration: Manipulating perceptual certainty of actions and their effects

Abstract

It has been proposed that statistical integration of multisensory cues may be a suitable framework to explain temporal binding, that is, the finding that causally related events such as an action and its effect are perceived to be shifted towards each other in time. A multisensory approach to temporal binding construes actions and effects as individual sensory signals, which are each perceived with a specific temporal precision. When they are integrated into one multimodal event, like an action-effect chain, the extent to which they affect this event's perception depends on their relative reliability. We test whether this assumption holds true in a temporal binding task by manipulating certainty of actions and effects. Two experiments suggest that a relatively uncertain sensory signal in such action-effect sequences is shifted more towards its counterpart than a relatively certain one. This was especially pronounced for temporal binding of the action towards its effect but could also be shown for effect binding. Other conceptual approaches to temporal binding cannot easily explain these results, and the study therefore adds to the growing body of evidence endorsing a multisensory approach to temporal binding.

Keywords: Multisensory processing; Perception and action; Temporal processing.

Fig. 1

Temporal signals employed in Experiment 1 and related binding predictions. a Types of actions and effects in Experiment 1. b Predicted binding effects in terms of statistical multisensory cue integration. The idealized likelihood functions illustrate the effect of changing certainty parameters and how they are expected to affect action and effect binding. Black curves show the expected distribution of temporal judgments of actions/effects in isolation, and gray curves show judgments of the same temporal signals in the respective contingent setting. In contrast, the motor and the mere causality approach both predict either no change of binding values between the conditions, if the manipulation has no influence on motor prediction or perceived causality, or similar change of action and effect binding values if there is such an influence

Fig. 2

Main results of Experiment 1. Action binding is shown to the left (red bars) and effect binding to the right (blue bars) ± SE of the mean (ms). Centrally, the judged timing in operant blocks of actions (red diamonds) and effects (blue diamonds) as well as judged timing in baseline blocks (white circles) are shown. *p < .05

Fig. 3

Types of actions and effects in Experiment 2

Fig. 4

Main results of Experiment 2. Action binding is shown to the left (red bars) and effect binding to the right (blue bars) ± SE of the corrected mean (ms). Centrally, the judged timing in operant blocks of actions (red diamonds) and effects (blue diamonds) as well as judged timing in baseline blocks (white circles) are shown. ^aSignificant only in a non-parametric test. *p < .05