Physiological Responses During Hybrid BNCI Control of an Upper-Limb Exoskeleton

Abstract

When combined with assistive robotic devices, such as wearable robotics, brain/neural-computer interfaces (BNCI) have the potential to restore the capabilities of handicapped people to carry out activities of daily living. To improve applicability of such systems, workload and stress should be reduced to a minimal level. Here, we investigated the user's physiological reactions during the exhaustive use of the interfaces of a hybrid control interface. Eleven BNCI-naive healthy volunteers participated in the experiments. All participants sat in a comfortable chair in front of a desk and wore a whole-arm exoskeleton as well as wearable devices for monitoring physiological, electroencephalographic (EEG) and electrooculographic (EoG) signals. The experimental protocol consisted of three phases: (i) Set-up, calibration and BNCI training; (ii) Familiarization phase; and (iii) Experimental phase during which each subject had to perform EEG and EoG tasks. After completing each task, the NASA-TLX questionnaire and self-assessment manikin (SAM) were completed by the user. We found significant differences (p-value < 0.05) in heart rate variability (HRV) and skin conductance level (SCL) between participants during the use of the two different biosignal modalities (EEG, EoG) of the BNCI. This indicates that EEG control is associated with a higher level of stress (associated with a decrease in HRV) and mental work load (associated with a higher level of SCL) when compared to EoG control. In addition, HRV and SCL modulations correlated with the subject's workload perception and emotional responses assessed through NASA-TLX questionnaires and SAM.

Keywords: Assistive technologies; brain-computer interfaces; exoskeleton.

Figure 1

Experimental setup: All the assistive technology shown in the figure was developed in the AIDE EU project framework to assist arm and hand function after severe paralysis. (a): Wearable eye tracker (Tobbi Glasses), electrooculography (EoG) interface, arm exoskeleton including pronosupination module and hand exoskelton. (b): A context-sensitive 3D-camera and visual feedback support precise and reliable assistance to grasp a real object; EEG and physiological signals monitoring system.

Figure 2

(Example of ERD and EOG control signals. (a) Example of event-related desynchronization (ERD) of “rest” and “motor imagery” (MI) phase. During calibration, discrimination threshold for reliable differentiation is set. (b) Overview of EOG control signals. Left side: EOG time series of horizontal eye movements (red). Movements are cued to left (blue long bars) and right (blue short bars) movements. Black dotted lines represent 60% discrimination thresholds based on maximum EOG peaks. Right side: Time-locked EOG signals of left/right horizontal eye movements during cue presentations. Plots show mean EOG with 95% confidence intervals (upper figures), which is calculated out of the EOG raw signals (lower figures).

Figure 3

Experimental phase: (a) Each subject has to perform two tasks: one of them triggered by EoG interface and the other triggered by EEG interface (each task lasts 6 min). After completing each task, questionnaires were submitted to the user. After completing the questionnaires and before starting with a task, the subjects remain in a relaxed state for 3 min to obtain baseline measurements; (b) When the EoG trigger is detected, the exoskeleton moves from rest position to target object position computed by means of the RGB-D camera. Once the object is reached and the EoG trigger is detected, the exoskeleton moves to rest position. (c) EEG signal is used to command the closing of the hand exoskeleton. Once the hand is closed, EEG trigger is required to open the hand exoskeleton.

Figure 4

Changes in physiological signals between interfaces. HRV and SCL show a significant difference (* $p<0.05$) between the two interfaces. Pulse rate shows a trend to be higher in EEG tasks related to EoG tasks.

Figure 5

Temporal evolution of each physiological signal (HRV, SCL, respiration and pulse rate). They are shown for each minute (1, 2, 3, 4, 5, and 6 min) and interface (EEG vs. EoG).

Figure 6

Normalized values of SAM and NASA-TLX tests for both interfaces (** p < 0.01; *** p < 0.001; **** p < 0.0001)