Augmentation of haptic feedback for teleoperated robotic surgery

Abstract

Purpose: A frequently mentioned lack of teleoperated surgical robots is the lack of haptic feedback. Haptics are not only able to mirror force information from the situs, but also to provide spatial guidance according to a surgical plan. However, superposition of the two haptic information can lead to overlapping and masking of the feedback and guidance forces. This study investigates different approaches toward a combination of both information and investigates effects on system usability.

Methods: Preliminary studies are conducted to define parameters for two main experiments. The two main experiments constitute simulated surgical interventions where haptic guidance as well as haptic feedback provide information for the surgeon. The first main experiment considers drilling for pedicle screw placements, while the second main experiment refers to three-dimensional milling tasks such as during partial knee replacements or craniectomies. For both experiments, different guidance modes in combination with haptic feedback are evaluated regarding effectiveness (e.g., distance to target depth), efficiency and user satisfaction (e.g., detectability of discrepancies in case of technical guidance error).

Results: Regarding pedicle screw placements a combination of a peripheral visual signal and a vibration constitutes a good compromise regarding distance to target depth and detectability of discrepancies. For milling tasks, trajectory guidance is able to improve efficiency and user satisfaction (e.g., perceived workload), while boundary constraints improve effectiveness. If, assistance cannot be offered in all degrees of freedom (e.g., craniectomies), a visual substitution of the haptic force feedback shows the best results, though participants prefer using haptic force feedback.

Conclusion: Our results suggest that in case haptic feedback and haptic assistance are combined appropriately, benefits of both haptic modalities can be exploited. Thereby, capabilities of the human-machine system are improved compared to usage of exclusively one of the haptic information.

Keywords: Haptics; Human machine interaction; Robotic manipulators; Shared control; Surgical robotics; Synergistic systems.

Fig. 1

Graphical use interface (GUI) for the preliminary study of stimulus sensation and reaction times (4° circle of foveal field of view only inserted to support understanding)

Fig. 2

Stimulus sensation (pleasantness—disturbance; maximum standard error ≈ 0.9) and reaction time in ms for the different signals and combinations (F foveal, P peripheral, A auditory, V vibration)

Fig. 3

Reaction time for different input modalities in ms

Fig. 4

Results of the questionnaire regarding the different input modalities (error bars indicate the standard error)

Fig. 5

Graphical user interface for the first main experiment regarding pedicle screw placements

Fig. 6

GUI for the second main experiment regarding milling; depth of the individual voxels is color-coded; background indicates active forces in depth direction (haptic guidance ≙ green, haptic feedback ≙ yellow, no force ≙ white)

Fig. 7

Process diagram of the second main experiment

Fig. 8

Distance to target depth in mm for drillings without discrepancies (*p < 0.05; **p < 0.01; ***p < 0.001) (F haptic feedback, B binary (vibration + peripheral), V visual (scale), T tactile (scale), W wall (kinesthetic + peripheral))

Fig. 9

Mean (absolute) deviation from goal depth in mm for milling (*p < 0.05; **p < 0.01; ***p < 0.001) (0 no haptic feedback, F haptic feedback, C constraint, T trajectory, S scaled, Vers versatile)

Fig. 10

Duration in s of milling (*p < 0.05; **p < 0.01; ***p < 0.001) (0 no haptic feedback, F haptic feedback, C constraint, T trajectory, S scaled, Vers versatile)

Fig. 11

User satisfaction for milling (*p < 0.05; **p < 0.01; ***p < 0.001) (0 no haptic feedback, F haptic feedback, C constraint, T trajectory, S scaled, Vers versatile)

Fig. 12

Exemplary superposition of all milling paths for P1 (partial knee replacement) (0 no haptic feedback, F haptic feedback, C constraint, T trajectory, S scaled, Vers versatile)