Movement Sonification: Effects on Motor Learning beyond Rhythmic Adjustments

Abstract

Motor learning is based on motor perception and emergent perceptual-motor representations. A lot of behavioral research is related to single perceptual modalities but during last two decades the contribution of multimodal perception on motor behavior was discovered more and more. A growing number of studies indicates an enhanced impact of multimodal stimuli on motor perception, motor control and motor learning in terms of better precision and higher reliability of the related actions. Behavioral research is supported by neurophysiological data, revealing that multisensory integration supports motor control and learning. But the overwhelming part of both research lines is dedicated to basic research. Besides research in the domains of music, dance and motor rehabilitation, there is almost no evidence for enhanced effectiveness of multisensory information on learning of gross motor skills. To reduce this gap, movement sonification is used here in applied research on motor learning in sports. Based on the current knowledge on the multimodal organization of the perceptual system, we generate additional real-time movement information being suitable for integration with perceptual feedback streams of visual and proprioceptive modality. With ongoing training, synchronously processed auditory information should be initially integrated into the emerging internal models, enhancing the efficacy of motor learning. This is achieved by a direct mapping of kinematic and dynamic motion parameters to electronic sounds, resulting in continuous auditory and convergent audiovisual or audio-proprioceptive stimulus arrays. In sharp contrast to other approaches using acoustic information as error-feedback in motor learning settings, we try to generate additional movement information suitable for acceleration and enhancement of adequate sensorimotor representations and processible below the level of consciousness. In the experimental setting, participants were asked to learn a closed motor skill (technique acquisition of indoor rowing). One group was treated with visual information and two groups with audiovisual information (sonification vs. natural sounds). For all three groups learning became evident and remained stable. Participants treated with additional movement sonification showed better performance compared to both other groups. Results indicate that movement sonification enhances motor learning of a complex gross motor skill-even exceeding usually expected acoustic rhythmic effects on motor learning.

Keywords: audiovisual information; motor learning; motor perception; motor rehabilitation; movement sonification; multisensory integration.

Figure 1

Experimental procedure: pre st, strength pretest; pre te, technique pretest; ts1-6, training session 1–6; post st, strength posttest; r, technique retention test.

Figure 2

Visual and auditory stimuli under the three conditions: frame of instruction video (above) + sound pressure level of soundtrack of a single rowing cycle (below); (A) group V, (B) group AV_nat, and (C) group AV_soni.

Figure 3

(A) Kinematic parameters; (B) Dynamic parameters. Data curve (up), amplitude level diagram (middle) and spectral graph (down) of the four data channels of a single rowing cycle (3.03 s).

Figure 4

The five training blocks of a single training session. One block is depicted magnified. IN, instruction; FB, feedback; c, cycles.

Figure 5

Formula to compute the general distance value (GDV). GF, grip force; FF, footrest forces; GP, grip pull-out; SS, sliding seat; FI, force index. Distance value of the grip force is weighted five-fold, other distance values one-way, half of force index is subtracted.

Figure 6

Learning curves, showing the development of group means of GDV from pretest to retention test for all three treatment groups. Pre, pretest; ts, training session; r, retention test; AV_soni, treatment group AV_soni; AV_nat, treatment group AV_nat; V, treatment group V. Standard deviations are regarded subsequently.

Figure 7

Course of GDV and its standard deviation from training session one to six (ts1–6), averaged for all blocks and all participants.

Figure 8

Course of GDV and its standard deviation during the five blocks of a training session (b1–5), averaged for all training sessions and all participants.

Figure 9

Group means and standard deviations of GDV for treatment group AV_soni, AV_nat, and V averaged for all participants and measurements.

Figure 10

Course of GDV within the six training sessions (ts1–6) averaged for all participants. b1–5: blocks 1–5.

Figure 11

Development of footrest forces from pretest to retention test for treatment group AV_soni, AV_nat, and V (group means of normalized data). Pre, pretest; ts, training session; r, retention test.

Figure 12

Development of duration of pull-out phase from pretest to retention test for treatment group AV_soni, AV_nat, and V (group means of normalized data). pre, pretest; ts, training session; r, retention test.

Figure 13

Development of Standard Deviations of GDV for the three treatment groups. ^xmark measuring points with significant results of Levene's Test for Homogeneity of Variances.

Figure 14

Changing of group means of maximum strength value (upper part with left ordinate) and development of group means of force index (FI) in the course of the training (lower part with right ordinate) from pretest (pre) via all blocks of all training sessions (ts1-6) to retention test (r).