Mind Wandering Influences EEG Signal in Complex Multimodal Environments

Abstract

The phenomenon of mind wandering (MW), as a family of experiences related to internally directed cognition, heavily influences vigilance evolution. In particular, humans in teleoperations monitoring partially automated fleet before assuming manual control whenever necessary may see their attention drift due to internal sources; as such, it could play an important role in the emergence of out-of-the-loop (OOTL) situations and associated performance problems. To follow, quantify, and mitigate this phenomenon, electroencephalogram (EEG) systems already demonstrated robust results. As MW creates an attentional decoupling, both ERPs and brain oscillations are impacted. However, the factors influencing these markers in complex environments are still not fully understood. In this paper, we specifically addressed the possibility of gradual emergence of attentional decoupling and the differences created by the sensory modality used to convey targets. Eighteen participants were asked to (1) supervise an automated drone performing an obstacle avoidance task (visual task) and (2) respond to infrequent beeps as fast as possible (auditory task). We measured event-related potentials and alpha waves through EEG. We also added a 40-Hz amplitude modulated brown noise to evoke steady-state auditory response (ASSR). Reported MW episodes were categorized between task-related and task-unrelated episodes. We found that N1 ERP component elicited by beeps had lower amplitude during task-unrelated MW, whereas P3 component had higher amplitude during task-related MW, compared with other attentional states. Focusing on parieto-occipital regions, alpha-wave activity was higher during task-unrelated MW compared with others. These results support the decoupling hypothesis for task-unrelated MW but not task-related MW, highlighting possible variations in the "depth" of decoupling depending on MW episodes. Finally, we found no influence of attentional states on ASSR amplitude. We discuss possible reasons explaining why. Results underline both the ability of EEG to track and study MW in laboratory tasks mimicking ecological environments, as well as the complex influence of perceptual decoupling on operators' behavior and, in particular, EEG measures.

Keywords: attentional decoupling; automation; mind wandering; out of the loop; sensory modalities; vigilance.

Figure 1

Experimental setup. The participant is equipped with the EEG system and sits in front of the right screen (LIPS screen). Speakers are on both sides of the right screen. The left screen is used to display attentional probes.

Figure 2

Screenshot of the LIPS interface. The plane in the center is static and the surrounding (yellow and red numbered symbols) are moving. During the left and right avoidance maneuver, again, the plane remains static and the background rotates.

Figure 3

Task-related and task-unrelated MW evolution through blocks. Error bars show the 95% CIs based on bootstrap.

Figure 4

Influence of blocks and attentional states on beep reaction time. Error bars show the 95% CIs based on bootstrap.

Figure 5

Beep ERP signal for task-related MW (green), task-unrelated MW (blue), and focus (red) attentional states.

Figure 6

Topography of alpha frequency for each attentional state.

Figure 7

Topography of ASSR frequency for each attentional state.

Figure 8

Spectrum of 35–45 Hz interval for each attentional state.

Figure 9

Spectrum of 0–45 Hz interval for each attentional state.