State of the Art in Immersive Interactive Technologies for Surgery Simulation: A Review and Prospective

Abstract

Immersive technologies have thrived on a strong foundation of software and hardware, injecting vitality into medical training. This surge has witnessed numerous endeavors incorporating immersive technologies into surgery simulation for surgical skills training, with a growing number of researchers delving into this domain. Relevant experiences and patterns need to be summarized urgently to enable researchers to establish a comprehensive understanding of this field, thus promoting its continuous growth. This study provides a forward-looking perspective by reviewing the latest development of immersive interactive technologies for surgery simulation. The investigation commences from a technological standpoint, delving into the core aspects of virtual reality (VR), augmented reality (AR) and mixed reality (MR) technologies, namely, haptic rendering and tracking. Subsequently, we summarize recent work based on the categorization of minimally invasive surgery (MIS) and open surgery simulations. Finally, the study showcases the impressive performance and expansive potential of immersive technologies in surgical simulation while also discussing the current limitations. We find that the design of interaction and the choice of immersive technology in virtual surgery development should be closely related to the corresponding interactive operations in the real surgical speciality. This alignment facilitates targeted technological adaptations in the direction of greater applicability and fidelity of simulation.

Keywords: augmented reality; interactive technology; mixed reality; surgery simulation; virtual reality.

Figure 1

Commonly used force-feedback device (PHANTOM^® Omni) in virtual surgery systems (please refer to [25,26] for more details on hardware specification).

Figure 2

Exemplary MIS simulation systems. (a) illustrates a virtual reality cholecystectomy simulator with a complete movable worktable [54]. (b) demonstrates a laparoscopic simulator with high-fidelity soft tissue rendering [55].

Figure 3

A mixed-reality-based prototype system for micro-anastomosis [11]. (a) depicts the workbench design of the system. The white arrows indicate various components in the workbench, including a head-mounted-display (HMD), several camera sensors, etc. (b) shows a simulation process of the microvascular hepatic artery reconstruction (MHAR) surgery by using the system.