Effects of an Audiovisual Emotion Perception Training for Schizophrenia: A Preliminary Study

Abstract

Individuals with schizophrenia show a reduced ability to integrate facial and vocal information in emotion perception. Although emotion perception has been a target for treatment, no study has yet examined the effect of multimodal training on emotion perception in schizophrenia. In the present study, we developed an audiovisual emotion perception training and test in which a voice and a face were simultaneously presented, and subjects were asked to judge whether the emotions of the voice and the face matched. The voices were either angry or happy, and the faces were morphed on a continuum ranging from angry to happy. Sixteen patients with schizophrenia participated in six training sessions and three test sessions (i.e., pre-training, post-training, and generalization). Eighteen healthy controls participated only in pre-training test session. Prior to training, the patients with schizophrenia performed significantly worse than did the controls in the recognition of anger; however, following the training, the patients showed a significant improvement in recognizing anger, which was maintained and generalized to a new set of stimuli. The patients also improved the recognition of happiness following the training, but this effect was not maintained or generalized. These results provide preliminary evidence that a multimodal, audiovisual training may yield improvements in anger perception for patients with schizophrenia.

Keywords: audiovisual; emotion perception; multimodal integration; schizophrenia; training.

Figure 1

(A) Trial sequences and morphed images used at the training and test and (B) schematic illustration of the procedure. SP, schizophrenia patient; HC, healthy control.

Figure 2

Emotion perception performances (d') of the SP and HC according to time (T₀, T₁, and T₂). Emotion perception performances for (A) anger, (B) happiness, and (C) total. d' = Z(hit rate) – Z(false alarm rate), where function Z(p), p ∈ [0,1]. Error bar indicates standard error. SP, schizophrenia patient; HC, healthy control. T₀, pre-training session; T₁, post-training session; T₂, generalization session.

Figure 3

(A) Pre-existing group differences and (B) training and generalization effects in SP group. d' = Z(hit rate) – Z(false alarm rate), where function Z(p), p ∈ [0,1]. Error bar indicates standard error. SP, schizophrenia patient; HC, healthy control. T₀, pre-training session; T₁, post-training session; T₂, generalization session.