Facial affect recognition in context in adults with and without TBI

Abstract

Introduction: Several studies have reported impaired emotion recognition in adults with traumatic brain injury (TBI), but studies have two major design features that limit application of results to real-world contexts: (1) participants choose from among lists of basic emotions, rather than generating emotion labels, and (2) images are typically presented in isolation rather than in context. To address these limitations, we created an open-labeling task with faces shown alone or in real-life scenes, to more closely approximate how adults with TBI label facial emotions beyond the lab.

Methods: Participants were 55 adults (29 female) with moderate to severe TBI and 55 uninjured comparison peers, individually matched for race, sex, and age. Participants viewed 60 photographs of faces, either alone or in the pictured person's real-life context, and were asked what that person was feeling. We calculated the percent of responses that were standard forced-choice-task options, and also used sentiment intensity analysis to compare verbal responses between the two groups. We tracked eye movements for a subset of participants, to explore whether gaze duration or number of fixations helped explain any group differences in labels.

Results: Over 50% of responses in both groups were words other than basic emotions on standard affect tasks, highlighting the importance of eliciting open-ended responses. Valence of labels by participants with TBI was attenuated relative to valence of Comparison group labels, i.e., TBI group responses were less positive to positive images and the same was true for negative images, although the TBI group responses had higher lexical diversity. There were no significant differences in gaze duration or number of fixations between groups.

Discussion: Results revealed qualitative differences in affect labels between adults with and without TBI that would not have emerged on standard forced-choice tasks. Verbal differences did not appear to be attributable to differences in gaze patterns, leaving open the question of mechanisms of atypical affect processing in adults with TBI.

Keywords: affect recognition; communication; emotion recognition; eye-tracking; facial expression; social cognition; speech-language pathology; traumatic brain injury.

Figure 1

Sample stimuli with face only and face in context. Image source: Interim Archives / RV Spencer. Reproduced with permission from Getty Images.

Figure 2

Boxplot showing participants’ mean sentiment scores for negatively valenced, neutral, and positively valenced images. Sentiment score means are shown in dark gray for the comparison group and light gray for the TBI group. ***p <0.001.

Figure 3

Boxplot depicting participants’ mean sentiment scores for negatively valenced, neutral, and positive images. Sentiment score means are shown in dark gray for males and light gray for females. ***p <0.001.