Inferring individual evaluation criteria for reaching trajectories with obstacle avoidance from EEG signals

Abstract

During reaching actions, the human central nerve system (CNS) generates the trajectories that optimize effort and time. When there is an obstacle in the path, we make sure that our arm passes the obstacle with a sufficient margin. This comfort margin varies between individuals. When passing a fragile object, risk-averse individuals may adopt a larger margin by following the longer path than risk-prone people do. However, it is not known whether this variation is associated with a personalized cost function used for the individual optimal control policies and how it is represented in our brain activity. This study investigates whether such individual variations in evaluation criteria during reaching results from differentiated weighting given to energy minimization versus comfort, and monitors brain error-related potentials (ErrPs) evoked when subjects observe a robot moving dangerously close to a fragile object. Seventeen healthy participants monitored a robot performing safe, daring and unsafe trajectories around a wine glass. Each participant displayed distinct evaluation criteria on the energy efficiency and comfort of robot trajectories. The ErrP-BCI outputs successfully inferred such individual variation. This study suggests that ErrPs could be used in conjunction with an optimal control approach to identify the personalized cost used by CNS. It further opens new avenues for the use of brain-evoked potential to train assistive robotic devices through the use of neuroprosthetic interfaces.

Figure 1

Experimental protocol. Participants directed the robot by using the joystick in their hand and evaluated a variety of robot trajectories with obstacle avoidance while collecting their EEG signals. Robot trajectories were generated by a dynamical system and modulated by two parameters, a safety factor s to control the distance to the obstacle and a reactivity factor $\rho$ to determine when the robot starts avoiding the obstacle. For each robot trajectory, we compute the energy efficiency, measured as the distance travelled by the robot, and the comfort, measured as the minimum distance between the end-effector and the obstacle, and relate this to individual specific preferences. We assess such personalized evaluation criteria by transferring individually calibrated classifier across participants.

Figure 2

Example robot trajectories. Sets of example robot trajectories with (a) low energy efficiency and high comfort, and (b, c) high energy efficiency and low comfort.

Figure 3

Electrophysiological results. (a) Time-frequency representation of error-related spectral perturbation (ERSP) at FCz within the time window of [$-0.5,1$] s with respect to onset of joystick release. (b) Theta band ([4 8] Hz) and mu band ([8 12] Hz) spectral power of all channels relative to the correct trials. (c) Grand-averaged event-related potentials of correct and erroneous trials at FCz. 0 s in x axis represents the onset of release for erroneous trials, while it corresponds to the individual averaged release time for correct trials. Insets illustrate scalp topographical representation of the obtained ErrPs at 0 and 0.25 s. (d) Classification performance measured by Area Under the Curve (AUC) for each subject. Each bar corresponds to the averaged AUC, while each dot corresponds to the AUC from one testing fold or averaged AUC score of single participants (in red).

Figure 4

Evaluation criteria with behavioral responses. (a) Mean and (b) variability of energy efficiency by comfort matrices based on behavioral responses. (c, d) Evaluation criteria of risk-prone and -averse subjects, respectively. A dashed black line shows the decision boundary between correct and error trials.

Figure 5

Inferred evaluation criteria based on ErrP. (a) Mean and (b) variability of energy efficiency by comfort matrices based on ErrP decoding output. (c, d) Inferred evaluation criteria based on ErrP detection for risk-prone and -averse subjects, respectively. A dashed black line shows the decision boundary between correct and error trials.

Figure 6

Individual customization of evaluation criteria. (a) Results of transfering the individually calibrated classifier between intra- and inter-subject data for two teaching labels; i.e., behavior and ErrP. ${}^{\ast }$ indicates $p<0.05$. (b) Correlation coefficient between evaluation criteria and inferred evaluation criteria of intra- and intersubject data. ${}^{\ast \ast \ast }$ represents the significant difference between the two groups (Wilcoxon’s signed-rank test, $p<$ 0.001).