Exploring the use of AI text-to-image generation to downregulate negative emotions in an expressive writing application

Abstract

Conventional writing therapies are versatile, accessible and easy to facilitate online, but often require participants to self-disclose traumatic experiences. To make expressive writing therapies safer for online, unsupervised environments, we explored the use of text-to-image generation as a means to downregulate negative emotions during a fictional writing exercise. We developed a writing tool, StoryWriter, that uses Generative Adversarial Network models to generate artwork from users' narratives in real time. These images were intended to positively distract users from their negative emotions throughout the writing task. In this paper, we report the outcomes of two user studies: Study 1 (N = 388), which experimentally examined the efficacy of this application via negative versus neutral emotion induction and image generation versus no image generation control groups; and Study 2 (N = 54), which qualitatively examined open-ended feedback. Our results are heterogeneous: both studies suggested that StoryWriter somewhat contributed to improved emotion outcomes for participants with pre-existing negative emotions, but users' open-ended responses indicated that these outcomes may be adversely modulated by the generated images, which could undermine the therapeutic benefits of the writing task itself.

Keywords: AI art; emotion regulation; image generation.

Figure 1.

System architecture of StoryWriter.

Figure 2.

User interface of StoryWriter.

Figure 3.

The flow of the experiment. Participants were first induced to feel a negative (versus neutral) emotion through an emotion induction task before using the StoryWriter application.

Figure 4.

The results of the StoryWriter/control application for regulating negative emotion. ‘*’ indicates p < 0.05, ‘**’ indicates p < 0.01 and ‘***’ indicates p < 0.001.

Figure 5.

The results of the StoryWriter/control applications for emotion improvement, usability, engagement and Telemetry. ‘*’ indicates p < 0.05 and ‘**’ indicates p < 0.01.

Figure 6.

Examples of written narratives and the respective images synthesized by the text-to-image DM-GAN model.

Figure 7.

Negative emotional trajectory of participants in the ES survey. Leftmost column lists the participants; centre columns indicate instances of the given emotion at each time point (greater saturation represents more instances); rightmost column indicates change in the given emotion between baseline and post-study time points (more green represents lower presence of given negative emotion, more red represents greater presence of given negative emotion). # Cases = number of participants who reported given emotion at each time point.

Figure 8.

Negative emotional trajectory of participants in the NS survey. Leftmost column lists the participants; centre columns indicate instances of the given emotion at each time point (greater saturation represents more instances); rightmost column indicates change in the given emotion between baseline and post-study time points (more green represents lower presence of given negative emotion, more red represents greater presence of given negative emotion). # Cases = number of participants who reported given emotion at each time point.

Figure 9.

Total emotional trajectory of participants in the ES survey. Leftmost column lists the participants; centre columns indicate the sum of given emotions at each time point (positive emotions as positive values, negative emotions as negative values, neutral emotions as zero value); rightmost column indicates change in total emotional valence between baseline and post-study time points (more green represents more positive emotion change, more red represents more negative emotion change). # Cases = number of participants who reported a given emotion at each time point.

Figure 10.

Total emotional trajectory of participants in the NS survey. Leftmost column lists the participants; centre columns indicate the sum of given emotions at each time point (positive emotions as positive values, negative emotions as negative values, neutral emotions as zero value); rightmost column indicates change in total emotional valence between baseline and post-study time points (more green represents more positive emotion change, more red represents more negative emotion change). # Cases = number of participants who reported a given emotion at each time point.

Figure 11.

Positive emotional trajectory of participants. Leftmost column lists the participants; centre columns indicate instances of the given emotion at each time point (greater saturation represents more instances); rightmost column indicates change in the given emotion between baseline and post-study time points (more green represents greater presence of given positive emotion, more red represents lower presence of given positive emotion). # Cases = number of participants who reported given emotion at each time point.