Wearable technology-based metrics for predicting operator performance during cardiac catheterisation

Abstract

Introduction: Unobtrusive metrics that can auto-assess performance during clinical procedures are of value. Three approaches to deriving wearable technology-based metrics are explored: (1) eye tracking, (2) psychophysiological measurements [e.g. electrodermal activity (EDA)] and (3) arm and hand movement via accelerometry. We also measure attentional capacity by tasking the operator with an additional task to track an unrelated object during the procedure.

Methods: Two aspects of performance are measured: (1) using eye gaze and psychophysiology metrics and (2) measuring attentional capacity via an additional unrelated task (to monitor a visual stimulus/playing cards). The aim was to identify metrics that can be used to automatically discriminate between levels of performance or at least between novices and experts. The study was conducted using two groups: (1) novice operators and (2) expert operators. Both groups made two attempts at a coronary angiography procedure using a full-physics virtual reality simulator. Participants wore eye tracking glasses and an E4 wearable wristband. Areas of interest were defined to track visual attention on display screens, including: (1) X-ray, (2) vital signs, (3) instruments and (4) the stimulus screen (for measuring attentional capacity).

Results: Experts provided greater dwell time (63% vs 42%, p = 0.03) and fixations (50% vs 34%, p = 0.04) on display screens. They also provided greater dwell time (11% vs 5%, p = 0.006) and fixations (9% vs 4%, p = 0.007) when selecting instruments. The experts' performance for tracking the unrelated object during the visual stimulus task negatively correlated with total errors (r = - 0.95, p = 0.0009). Experts also had a higher standard deviation of EDA (2.52 µS vs 0.89 µS, p = 0.04).

Conclusions: Eye tracking metrics may help discriminate between a novice and expert operator, by showing that experts maintain greater visual attention on the display screens. In addition, the visual stimulus study shows that an unrelated task can measure attentional capacity. Trial registration This work is registered through clinicaltrials.gov, a service of the U.S. National Health Institute, and is identified by the trial reference: NCT02928796.

Keywords: Attentional capacity; Eye tracking; Simulation-based training; Surgical simulation; Wearable technology.

Fig. 1

Main image: Mentice VIST-Lab simulator, with the four AOIs identified. Bottom right: a participant during procedural performance, wearing eye tracking glasses connected to the portable recording device placed to the left on the simulator table and wearing the Empatica’s E4 wristband on their wrist (hidden)

Fig. 2

Card acknowledgement % effect on total errors for first attempt. (1) All participants (full dataset), (2) novice only, (3) expert only

Fig. 3

Card acknowledgement % relationship with total errors for the final attempt. (1) All participants (full dataset) included, (2) a clear outlier (a novice) is removed from dataset, (3) novice only, (4) novice only with outlier removed, (5) expert only

Fig. 4

Group comparison for transition frequency over all AOIs

Fig. 5

Group comparison of calculated SD for recorded EDA during both attempts