Experiencing art together: integrating affect and semiosis

Abstract

Introduction: Art is ubiquitous in our lives, and its experience and understanding are deeply emotional. Dewey suggested that all human experience, including art experiences, emerges from active engagement with the environment. In this view, affect and interpretation are interconnected processes that unfold together. To examine the integration of these processes, this interdisciplinary study used a multi-method approach.

Methods: Eighteen dyads of adult participants took part in the study. They were instructed to each bring an art object that was meaningful to them. During the experiment participants engaged in an audio-visually recorded, semi-structured conversation, reflecting on both art objects. They also answered pre- and post-questionnaires on their emotions. Affect was measured through self-reported valence and arousal of emotions, and sentiment analysis of the conversation. Semiosis as the process of making sense of the art objects was operationalized in terms of four strategies, namely: perception, imagination, conceptualization, and analysis. Affect was measured through self-reported valence and arousal of emotions, and sentiment analysis of the conversation.

Results and discussion: The results showed that dyadic interactions led to changes, at the group level, in participants' self-reported affect toward the other's art object. An Exploratory Graph Analysis revealed unique weighted networks of sentiment for each strategy. Additionally, a Multinomial Log-linear Model demonstrated that affect and strategies work in tandem during the art experience, to predict perceived affect.

Keywords: affect; art experience; exploratory graph analysis; semiosis; sentiment analysis.

FIGURE 1

The proposed model. Art experience can occur at both individual and collective levels, incorporating affective elements (respectively, sentiment represented as yellow circles and perceived affect as pins) and semiotic strategies (represented as violet circles) that interact dynamically (shown as lines). The model includes “checkpoints” –moments within the process that occur above our threshold of awareness and manifest as perceived affect (represented as pins). This interaction of affective aspects and semiotic strategies shapes the experience itself, creating a feedback loop that informs and refines future experiences.

FIGURE 2

Overview of data collection process. During the Experimental Phase, ‘Experience’ refers to the participant’s engagement with the art objects through the allowed modalities (e.g.: listening, observing, touching, smelling, watching, etc.).

FIGURE 3

Dutch-English Geneva Emotion Wheel, its quadrants and information for calculations. The wheel was within a pixel grid of 900 × 632, and it has a diameter of 496 pixels, cantered at coordinates (465, 316). The red dot represents an example point at the measure that can be derived from its xy-coordinates. The wheel organizes emotions within a 2D space based on the circumflex model of emotion (Russell, 1980). This allows us to infer two dimensions: Valence (represented on the x-axis) and Arousal (represented on the y-axis). Clicks are positioned in specific quadrants within the wheel, making it possible to calculate these dimensions using the x and y coordinates of each selected emotion. For example, a click located on the right side of the wheel indicates a positive valence, and, if the click is higher on the y-axis it suggests higher arousal.

FIGURE 4

Contour plot of distribution of points in Geneva Emotion Wheel (GEW).

FIGURE 5

EGA plot of latent communities of sentiment across strategies. Nodes are positioned based on the eigenstructure of the adjacency matrix (Golino and Christensen, 2019), which organizes them according to the strength and pattern of their connections. This layout brings closely related nodes nearer to each other.

FIGURE 6

Means for probabilities (0-1) of sentiment across strategies.