Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Abstract

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Keywords: EEG; assistive robots; autonomous interactive systems; cognitive workload classification; eye gaze; multi-modality learning; pupillometry.

Figure 1

Pupillometry pre-processing steps: (a) raw pupil diameter signal, (b) signal after applying amplitude thresholding, (c) signal after applying linear interpolation, and (d) signal after applying Butterworth low-pass filter.

Figure 2

Arterial blood pressure (ABP) signal pre-processing and SP/DP point detection steps: (a) raw ABP signal, (b) signal after upsampling and applying law-pass filter, (c) the candidate SP/DP points, and (d) estimated SP/DP values.

Figure 3

The variations in APCPS obtained from eight samples and the variation in mean APCPS over 116 samples (upper left) for three different workload levels.

Figure 4

Boxplot of mean APCPS over all events for different cognitive workload levels.

Figure 5

Analysis of HRV time domain and frequency domain features for different workload levels.

Figure 6

Analysis of BPV time domain and frequency domain features for different workload levels.