When a model gives you mixed signals: cognitive effects and visual behavior

Abstract

Ambiguity in business process models can result in multiple interpretations by model readers. This leads to undesirable outcomes such as misunderstandings, unclear allocation of responsibilities, and unexpected behaviors. Despite these potential consequences, the impact of ambiguity on model readers has received limited attention so far. This article presents an eye-tracking study designed to investigate the effects of various types of ambiguity (i.e., layout, semantic, syntactic, and lexical) on readers' cognitive load, comprehension, and visual associations while interpreting process models. In addition, the study delves into the behaviors of model readers when resolving ambiguity in process models. These behaviors are investigated following a qualitative approach combining both eye-tracking and think-aloud data. The results demonstrate that ambiguities significantly influence cognitive load, comprehension, and visual associations, emphasizing the negative effects of ambiguity. Moreover, the qualitative insights suggest that participants exhibit specific behaviors when trying to resolve ambiguities. These findings underscore the need for advanced mechanisms to detect and mitigate ambiguity in process models.

Keywords: Ambiguity; Cognitive load; Eye-tracking; Process models; Visual behavior.

Fig. 1

BPMN fragment of a process with layout ambiguity

Fig. 2

BPMN fragments of processes with semantic (left) and syntactic (right) ambiguities

Fig. 3

Research model to answer RQ1. T = theoretical construct; O = operationalization of the construct

Fig. 4

Experiment procedure

Fig. 5

Overview of the qualitative exploratory analysis

Fig. 6

Example of a simplified AOIs-order over time plot

Fig. 7

Process maps comparing the visual associations of a participant (SP7) when reading a sub-process without (left) and with (right) a semantic ambiguity. A higher resolution of this figure is available in the online appendix. The circle with a dot inside denotes the process start, the double circle with a square inside denotes the process end. Rectangles refer to visits to the different process model activities; edges refer to the transitions for visiting one activity from another. The color scale of the rectangles refers to the absolute visit frequency to an activity; the thickness and labels on the edges refer to the absolute transition frequency, resp. the number of transitions between each pair of activities

Fig. 8

Excerpts from ambiguous process models showing the ambiguous model fragments. The complete models are available in the online appendix

Fig. 9

AOIs-order over time plot showing the Sweep visual behavior of participant SP7 while trying to resolve an ambiguity, visible in the sequence of short visits () to multiple process elements shown in different colors between the first and the last visits to the ambiguity (shown in black)

Fig. 10

AOIs-order over time plot showing the Explore visual behavior of participant SP6 while trying to resolve an ambiguity, visible in the sequence of long visits () to multiple process elements shown in different colors between the first and the last visits to the ambiguity (shown in black)

Fig. 11

AOIs-order over time plot showing the Target visual behavior of participant KP23 while trying to resolve an ambiguity, visible in the sequence of long visits () to the few process elements shown in yellow, green and violet between the first and the last visits to the ambiguity (shown in black)

Fig. 12

AOIs-order over time plot showing the Bounce visual behavior of participant SP20 while trying to resolve an ambiguity, visible in the alternating visits to the ambiguous process elements (shown in black) and another process element (shown in turquoise)

Fig. 13

AOIs-order over time plot showing the Hover visual behavior of participant SP9 while trying to resolve an ambiguity, visible in the (mostly) uninterrupted sequence of multiple long visits to the ambiguous process elements (shown in black)

Fig. 14

AOIs-order over time plot showing the interleaved visual behavior between Hover and Target by SP3 while trying to resolve an ambiguity

Fig. 15

Visual behavior codes in a scale of attention. Hover suggests the most focused attention when facing ambiguity, while Sweep suggests the most random attention