Virtual and augmented reality in biomedical engineering

Abstract

Background: In the future, extended reality technology will be widely used. People will be led to utilize virtual reality (VR) and augmented reality (AR) technologies in their daily lives, hobbies, numerous types of entertainment, and employment. Medical augmented reality has evolved with applications ranging from medical education to picture-guided surgery. Moreover, a bulk of research is focused on clinical applications, with the majority of research devoted to surgery or intervention, followed by rehabilitation and treatment applications. Numerous studies have also looked into the use of augmented reality in medical education and training.

Methods: Using the databases Semantic Scholar, Web of Science, Scopus, IEEE Xplore, and ScienceDirect, a scoping review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria. To find other articles, a manual search was also carried out in Google Scholar. This study presents studies carried out over the previous 14 years (from 2009 to 2023) in detail. We classify this area of study into the following categories: (1) AR and VR in surgery, which is presented in the following subsections: subsection A: MR in neurosurgery; subsection B: spine surgery; subsection C: oral and maxillofacial surgery; and subsection D: AR-enhanced human-robot interaction; (2) AR and VR in medical education presented in the following subsections; subsection A: medical training; subsection B: schools and curriculum; subsection C: XR in Biomedicine; (3) AR and VR for rehabilitation presented in the following subsections; subsection A: stroke rehabilitation during COVID-19; subsection B: cancer and VR, and (4) Millimeter-wave and MIMO systems for AR and VR.

Results: In total, 77 publications were selected based on the inclusion criteria. Four distinct AR and/or VR applications groups could be differentiated: AR and VR in surgery (N = 21), VR and AR in Medical Education (N = 30), AR and VR for Rehabilitation (N = 15), and Millimeter-Wave and MIMO Systems for AR and VR (N = 7), where N is number of cited studies. We found that the majority of research is devoted to medical training and education, with surgical or interventional applications coming in second. The research is mostly focused on rehabilitation, therapy, and clinical applications. Moreover, the application of XR in MIMO has been the subject of numerous research.

Conclusion: Examples of these diverse fields of applications are displayed in this review as follows: (1) augmented reality and virtual reality in surgery; (2) augmented reality and virtual reality in medical education; (3) augmented reality and virtual reality for rehabilitation; and (4) millimeter-wave and MIMO systems for augmented reality and virtual reality.

Keywords: AR in education; AR surgery; Augmented reality; Biomedical context; Virtual reality.

Fig. 1

MR, AR, and VR [4]

Fig. 2

Comparison between traditional image-guided surgery (A) and AR-guided surgery (B) [18]

Fig. 3

Image-to-patient registration strategies in OCMS. A In traditional navigated OCMS, a skull-fixed marker, here using retro-reflective spheres for tracking with an infrared sensor, is used for registration. B Alternatively, using patient-specific occlusal splints for non-invasive marker fixation has been proposed. C Marker-less registration, for example, using a laser scanner to capture the outer anatomy of the patient has also been suggested. However, to track the movement of the patient, a marker is still required [18]

Fig. 4

Approaches to augmenting reality in robotics [24]

Fig. 5

AR being used on a manikin [6]

Fig. 6

The AR app “HoloHuman” shows a virtual cadaver placed on a real examination table [33]

Fig. 7

HoloPatient system [33]

Fig. 8

The VR headset decreases the anxiety of the intensive care patient [65]

Fig. 9

The decentralized system architecture in AR [71]

Fig. 10

A proposed reference architecture of BC-based 6 G-envisioned massive IoT-supported AR/VR ecosystem [71]

Fig. 11

AR and VR in biomedical engineering: a review classification

Fig. 12

Technologies merged with AR and VR [17]