Multimodal Approach to Assess a Virtual Reality-based Surgical Training Platform

Abstract

Virtual reality (VR) can bring numerous benefits to the learning process. Combining a VR environment with physiological sensors can be beneficial in skill assessment. We aim to investigate trainees' physiological (ECG) and behavioral differences during the virtual reality-based surgical training environment. Our finding showed a significant association between the VR-Score and all participants' total NASA-TLX workload score. The extent of the NASA-TLX workload score was negatively correlated with VR-Score (R² =0.15, P < 0.03). In time-domain ECG analysis, we found that RMSSD (R² =0.16, P < 0.05) and pNN50 (R² =0.15, P < 0.05) scores correlated with significantly higher VR-score of all participants. In this study, we used SVM (linear kernel) and Logistic Regression classification techniques to classify the participants as gamers and non-gamers using data from VR headsets. Both SVM and Logistic Regression accurately classified the participants as gamers and non-gamers with 83% accuracy. For both SVM and Linear Regression, precision was noted as 88%, recall as 83%, and f1-score as 83%. There is increasing interest in characterizing trainees' physiological and behavioral activity profiles in a VR environment, aiming to develop better training and assessment methodologies.

Keywords: ECG; Mental workload; Skill assessment; Virtual Reality.

Fig. 1.

(a) VR-ECG Setup and (b) VR Racket Game

Fig. 2.

(a) Mean Position, (b) Mean Path Length, (c) Mean Velocity, (d) Mean Acceleration, and (e) Mean Jerk box plots comparing Gamer and Non-gamer results.

Fig. 3.

Clustering Results according to the gaming experience

Fig. 4.

Confusion matrix for (a) Logistic Regression, (b) SVM Linear Kernel, and (c) AdaBoost classifier

Fig. 5.

(a)VR Performance of non-gamer students (gray, circle indicates the median) vs. gamer students (black). Error bars indicate sample standard deviations (*p<0.05; **p< 0.01). (b) Nasa-TLX Total Score vs. VR Score

Fig. 6.

(a) VR score vs RMSSD (R² = 0.14 P<0.05). (b) VR Score vs PNN50 (R² = 0.16 P<0.05). In the scatter plots, the solid black lines indicate the linear best fit to the data points, and the dotted lines indicate the 95% confidence interval.