How do schizophrenia patients use visual information to decode facial emotion?

Abstract

Impairment in recognizing facial emotions is a prominent feature of schizophrenia patients, but the underlying mechanism of this impairment remains unclear. This study investigated the specific aspects of visual information that are critical for schizophrenia patients to recognize emotional expression. Using the Bubbles technique, we probed the use of visual information during a facial emotion discrimination task (fear vs. happy) in 21 schizophrenia patients and 17 healthy controls. Visual information was sampled through randomly located Gaussian apertures (or "bubbles") at 5 spatial frequency scales. Online calibration of the amount of face exposed through bubbles was used to ensure 75% overall accuracy for each subject. Least-square multiple linear regression analyses between sampled information and accuracy were performed to identify critical visual information that was used to identify emotional expression. To accurately identify emotional expression, schizophrenia patients required more exposure of facial areas (i.e., more bubbles) compared with healthy controls. To identify fearful faces, schizophrenia patients relied less on bilateral eye regions at high-spatial frequency compared with healthy controls. For identification of happy faces, schizophrenia patients relied on the mouth and eye regions; healthy controls did not utilize eyes and used the mouth much less than patients did. Schizophrenia patients needed more facial information to recognize emotional expression of faces. In addition, patients differed from controls in their use of high-spatial frequency information from eye regions to identify fearful faces. This study provides direct evidence that schizophrenia patients employ an atypical strategy of using visual information to recognize emotional faces.

Fig. 1.

Illustration of How the Bubbles Stimuli Were Generated. First, each of original faces was decomposed into 5 spatial frequency bandwidths of one octave each (the first row). Second, each spatial frequency bandwidth was independently sampled with randomly positioned Gaussian apertures or bubbles (the second row). Third, randomly sampled Gaussian apertures were applied to a decomposed face image at each spatial frequency to reveal partial facial information at each spatial frequency (the third row). The sampled information across spatial frequencies (plus a constant, nonsampled, and coarsest sixth spatial frequency) was combined to produce an experimental stimulus (the rightmost image in the third row) at a given trial.

Fig. 2.

Classification Images that Reveal Visual Facial Information Significantly Correlated with Correct Identification of Fear (A) and Happiness (B). For an illustrative purpose, the classification images were overlaid with faces decomposed at each spatial frequency indicated by the numbers above the images. The areas in red indicate facial features that healthy controls utilized significantly to identity fear (A) and happy (B). The areas in green show facial features that schizophrenia patients made use of to identity fear (A) and happy (B). The areas in yellow indicate facial features that both healthy controls and schizophrenia patients used.

Fig. 3.

Images that Summed Potent Visual Information across Spatial Frequencies for Correct Identification of Emotional Expressions: (A) fear in healthy controls, (B) fear in schizophrenia patients, (C) happy in healthy controls, and (D) happy in schizophrenia patients.