Effectiveness of Immersive Virtual Reality on Orthopedic Surgical Skills and Knowledge Acquisition Among Senior Surgical Residents: A Randomized Clinical Trial

Abstract

Importance: Video learning prior to surgery is common practice for trainees and surgeons, and immersive virtual reality (IVR) simulators are of increasing interest for surgical training. The training effectiveness of IVR compared with video training in complex skill acquisition should be studied.

Objectives: To evaluate whether IVR improves learning effectiveness for surgical trainees and to validate a VR rating scale through correlation to real-world performance.

Design, setting, and participants: This block randomized, intervention-controlled clinical trial included senior (ie, postgraduate year 4 and 5) orthopedic surgery residents from multiple institutions in Canada during a single training course. An intention-to-treat analysis was performed. Data were collected from January 30 to February 1, 2020.

Intervention: An IVR training platform providing a case-based module for reverse shoulder arthroplasty (RSA) for advanced rotator cuff tear arthropathy. Participants were permitted to repeat the module indefinitely.

Main outcomes and measures: The primary outcome measure was a validated performance metric for both the intervention and control groups (Objective Structured Assessment of Technical Skills [OSATS]). Secondary measures included transfer of training (ToT), transfer effectiveness ratio (TER), and cost-effectiveness (CER) ratios of IVR training compared with control. Additional secondary measures included IVR performance metrics measured on a novel rating scale compared with real-world performance.

Results: A total of 18 senior surgical residents participated; 9 (50%) were randomized to the IVR group and 9 (50%) to the control group. Participant demographic characteristics were not different for age (mean [SD] age: IVR group, 31.1 [2.8] years; control group, 31.0 [2.7] years), gender (IVR group, 8 [89%] men; control group, 6 [67%] men), surgical experience (mean [SD] experience with RSA: IVR group, 3.3 [0.9]; control group, 3.2 [0.4]), or prior simulator use (had experience: IVR group 6 [67%]; control group, 4 [44%]). The IVR group completed training 387% faster considering a single repetition (mean [SD] time for IVR group: 4.1 [2.5] minutes; mean [SD] time for control group: 16.1 [2.6] minutes; difference, 12.0 minutes; 95% CI, 8.8-14.0 minutes; P < .001). The IVR group had significantly better mean (SD) OSATS scores than the control group (15.9 [2.5] vs 9.4 [3.2]; difference, 6.9; 95% CI, 3.3-9.7; P < .001). The IVR group also demonstrated higher mean (SD) verbal questioning scores (4.1 [1.0] vs 2.2 [1.7]; difference, 1.9; 95% CI, 0.1-3.3; P = .03). The IVR score (ie, Precision Score) had a strong correlation to real-world OSATS scores (r = 0.74) and final implant position (r = 0.73). The ToT was 59.4%, based on the OSATS score. The TER was 0.79, and the system was 34 times more cost-effective than control, based on CER.

Conclusions and relevance: In this study, surgical training with IVR demonstrated superior learning efficiency, knowledge, and skill transfer. The TER of 0.79 substituted for 47.4 minutes of operating room time when IVR was used for 60 minutes.

Trial registration: ClinicalTrials.gov Identifier: NCT04404010.

Figure 1.. Study Flow Diagram

Figure 2.. Representative Cadaveric Specimen

D, Image uses different cadaveric specimen than other panels.

Figure 3.. Immersive Virtual Reality Operating Room and Metric Feedback

Figure 4.. Fourteen Representative Studies of Early Learning Curves and Composite Major Complication Rates for Performing Reverse Shoulder Arthroplasty

A total of 14 studies^{,,,,,,,,,,,,,} were used. A fitted curve (r² = 0.80) allows for determination of early learning curve reduction based on transfer of training ratios. In 1 study, multiple complications occurred in operative participants, resulting in a complication rate of greater than 100%. Our transfer of training of 32.5% to 59.4% illustrates that immersive virtual reality training reduces early learning curves by 13 to 51 cases, represented by the gray rectangle and solid vertical lines.