Experimental pilot study for augmented reality-enhanced elbow arthroscopy

Abstract

The purpose of this study was to develop and evaluate a novel elbow arthroscopy system with superimposed bone and nerve visualization using preoperative computed tomography (CT) and magnetic resonance imaging (MRI) data. We obtained bone and nerve segmentation data by CT and MRI, respectively, of the elbow of a healthy human volunteer and cadaveric Japanese monkey. A life size 3-dimensional (3D) model of human organs and frame was constructed using a stereo-lithographic 3D printer. Elbow arthroscopy was performed using the elbow of a cadaveric Japanese monkey. The augmented reality (AR) range of error during rotation of arthroscopy was examined at 20 mm scope-object distances. We successfully performed AR arthroscopy using the life-size 3D elbow model and the elbow of the cadaveric Japanese monkey by making anteromedial and posterior portals. The target registration error was 1.63 ± 0.49 mm (range 1-2.7 mm) with respect to the rotation angle of the lens cylinder from 40° to - 40°. We attained reasonable accuracy and demonstrated the operation of the designed system. Given the multiple applications of AR-enhanced arthroscopic visualization, it has the potential to be a next-generation technology for arthroscopy. This technique will contribute to the reduction of serious complications associated with elbow arthroscopy.

Figure 1

A real size 3-dimensional (3D) model of organs and frame. The model was constructed using a standard triangulated language (STL) 3D printer (Object500 Connex, Stratasys Ltd, US.).

Figure 2

Elbow arthroscopy and tracking device system. Red arrows indicate Xpoints (a) The Schema of augmented reality (AR) arthroscopy system. This figure was designed by Dr. Shintaro Oyama using Adobe Illustrator CS5 software (https://www.adobe.com/jp) (b).

Figure 3

Reverse distortion correction using lens distortion matrix. White arrows show differences before and after correction. Appropriate distortion of the shape on the monitor was corrected.

Figure 4

Elbow of a Japanese monkey cadaver and Standard Triangulated Language (STL) data. We used the elbow (1/2 of upper arm ~ 1/2 of forearm) of a Japanese monkey cadaver. From X-ray computed tomography (CT) data of the cadaveric elbow, we modeled frame data that can be precisely attached to humerus and ulna on the posture of 90° elbow flexion, 0° forearm pronation (a). We obtained bone segmentation from CT data and nerves from magnetic resonance imaging (MRI) data of a cadaveric Japanese monkey elbow. Segmentation and refinement were performed using VoTracer software (Riken, Wako, Japan) (http://www2.riken.jp/brict/Ijiri/VoTracer). This figure was generated by Dr. Shintaro Oyama using L-Phinus V5 Ver. 5.01 software (https://l-phinus.jp/software.html) (b).

Figure 5

Augmented reality (AR) system error calculation during arthroscopy rotation. Markers were installed on the camera head and lens cylinder. The working distance was set to 20 mm (Fig. 5a). A circular model with a diameter of 2 mm was used as the 3D model, and a checkerboard was used as the object (Fig. 5b). The target registration error was obtained from the superimposed display to evaluate the position accuracy. The angle of the lens cylinder was changed by 10° within the measurable range (-40° < θ < 40°) of MicronTracker 3 (Fig. 5c). Figure 5a–c were designed by Syuto Otsuka using PowerPoint for Microsoft 365.

Figure 6

Augmented reality (AR) arthroscopy on cadaveric Japanese monkey elbow. Capitellum and radial head were visualized through the anteromedial portal and visualized on the scope monitor (a). The humeroradial joint and radial nerve (white arrows) were superimposed on the real view (b). The red arrow indicates a third person view using the stereo camera on the optical tracking device.

Figure 7

The target registration error. The target registration error was 1.63 ± 0.49 mm (range, 1–2.7 mm) with respect to the rotation angle of the lens cylinder from 40° to − 40°.