Effect before cause: supramodal recalibration of sensorimotor timing

Abstract

Background: Our motor actions normally generate sensory events, but how do we know which events were self generated and which have external causes? Here we use temporal adaptation to investigate the processing stage and generality of our sensorimotor timing estimates.

Methodology/principal findings: Adaptation to artificially-induced delays between action and event can produce a startling percept--upon removal of the delay it feels as if the sensory event precedes its causative action. This temporal recalibration of action and event occurs in a quantitatively similar manner across the sensory modalities. Critically, it is robust to the replacement of one sense during the adaptation phase with another sense during the test judgment.

Conclusions/significance: Our findings suggest a high-level, supramodal recalibration mechanism. The effects are well described by a simple model which attempts to preserve the expected synchrony between action and event, but only when causality indicates it is reasonable to do so. We further demonstrate that this model successfully characterises related adaptation data from outside the sensorimotor domain.

Figure 1. Temporal order and synchronous/asynchronous judgments after adaptation to sensorimotor delay.

Raw data for representative, naïve observer KJW. (A) A sample of the psychometric functions generated via temporal order judgments: ‘which came first, action or event?’. This sample shows unimodal data from the audio-motor section of the adaptation experiment. The percentage of ‘event before action’ trials (i.e. where observers report an illusory reversal of temporal order) is plotted as a function of test delay (the physical asynchrony between their action (a mousepress) and a sensory event (an auditory ‘click’ in this example)). The different coloured functions represent different sensorimotor adaptation delays (see Figure key for details). (B) Data for the same observer for a control experiment. The only difference between the data shown in A&B is the nature of the judgment type. In this data set, observers were presented with the same stimuli (‘mousepress-click’) but made synchronous/asynchronous (as opposed to temporal order) judgements (see main text for details). The same Figure key applies to both panels.

Figure 2. Adaption shifts the point of subjective sensorimotor simultaneity across the sensorimotor pairings.

Temporal recalibration of motor action and a sensory event in the visual (red circles), auditory (blue squares) and tactile (green diamonds) sensorimotor pairings. Data points represent the physical sensorimotor asynchrony necessary to produce perceptual synchrony (PSS). Positive values signify a temporal lead of action over event. The data are fitted with a model with two free parameters (see main text for details). Error bars represent one standard error of the mean (variance between observers) either side of the parameter values (n = 5).

Figure 3. Temporal recalibration transfers to un-adapted sensorimotor pairings.

Average PSS values (ms) from (A) visuo-motor (V), (B) auditory-motor (A) and (c) tactile-motor (T) sensorimotor temporal order judgments following adaptation to a 200 ms delay between action and event. Positive values signify a temporal lead of action over event. Yellow bars represent within-modality data taken from the 200 ms condition in Figure 2. Grey bars represent the crossmodal conditions for the same delay. Error bars represent one standard error of the mean either side of the parameter values (n = 7).

Figure 4. Sensitivity to changes in relative sensorimotor time.

Average threshold values (ms) from (A) visuo-motor, (B) audio-motor and (C) tactile-motor sensorimotor temporal order judgments following adaptation to a 200 ms delay between action and event. In each plot, yellow bars represent within-modality data taken from the 200 ms condition in Figure 2. Grey bars represent crossmodal data (e.g., adapt visuo-motor, test audio-motor). Error bars represent one standard error of the mean either side of the parameter values (n = 7).

Figure 5. Adaptation to purely sensory asynchrony.

Data taken from Fujisaki at al. (2004) where observers adapted to a fixed level of asynchrony between auditory and visual stimulus pairs before judging the relative temporal relationship of audiovisual test pairs . PSS values are plotted as a function of the size of the adapting asynchrony and are expressed relative to the ‘adapt synchronous’ condition. The data are fitted with a same model used to fit the sensorimotor data shown in Figure 2 (see main text for details).