Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Sep 14;5(5):162-166.
doi: 10.1049/htl.2018.5072. eCollection 2018 Oct.

Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference

Rafael Moreta-Martinez  1   2 , David García-Mato  1   2 , Mónica García-Sevilla  1   2 , Rubén Pérez-Mañanes  2   3   4 , José Calvo-Haro  2   3   4 , Javier Pascau  1   2
Affiliations

Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference

Rafael Moreta-Martinez et al. Healthc Technol Lett. .

Abstract

Augmented reality (AR) can be an interesting technology for clinical scenarios as an alternative to conventional surgical navigation. However, the registration between augmented data and real-world spaces is a limiting factor. In this study, the authors propose a method based on desktop three-dimensional (3D) printing to create patient-specific tools containing a visual pattern that enables automatic registration. This specific tool fits on the patient only in the location it was designed for, avoiding placement errors. This solution has been developed as a software application running on Microsoft HoloLens. The workflow was validated on a 3D printed phantom replicating the anatomy of a patient presenting an extraosseous Ewing's sarcoma, and then tested during the actual surgical intervention. The application allowed physicians to visualise the skin, bone and tumour location overlaid on the phantom and patient. This workflow could be extended to many clinical applications in the surgical field and also for training and simulation, in cases where hard body structures are involved. Although the authors have tested their workflow on AR head mounted display, they believe that a similar approach can be applied to other devices such as tablets or smartphones.

Keywords: actual surgical intervention; augmented reality; automatic registration; bone; cancer; computer-assisted interventions; computerised tomography; extraosseous Ewing sarcoma; image registration; medical image processing; paediatrics; patient-specific 3D printed reference; patient-specific tools; placement errors; real-world spaces; software application; surgery; surgical field; tumour location; tumours; visual pattern.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Anatomical 3D models and axial CT slice displayed on HoloLens application
Fig. 2
Fig. 2
Surgical guide containing visual marker
Fig. 3
Fig. 3
Point recording on phantom a Virtual view b Real view c AR view
Fig. 4
Fig. 4
Surgical guide placement error for three repetitions
Fig. 5
Fig. 5
AR point localisation error for three users
Fig. 6
Fig. 6
Surgeon placing surgical guide in patient's tibia a HoloLens b Surgical guide
See this image and copyright information in PMC

References

    1. Inoue D., Cho B., Mori M., et al. : ‘Preliminary study on the clinical application of augmented reality neuronavigation’, J. Neurol. Surg. A. Cent. Eur. Neurosurg., 2013, 74, pp. 71–76 (doi: 10.1055/s-0032-1333415) - PubMed
    1. Coles T.R., John N.W., Gould D., et al. : ‘Integrating haptics with augmented reality in a femoral palpation and needle insertion training simulation’, IEEE Trans. Haptics, 2011, 4, (3), pp. 199–209 (doi: 10.1109/TOH.2011.32) - PubMed
    1. Abhari K., Baxter J., Chen E., et al. : ‘Training for planning tumour resection: augmented reality and human factors’, IEEE Trans. Biomed. Eng., 2015, 62, (6), pp. 1466–1477 (doi: 10.1109/TBME.2014.2385874) - PubMed
    1. Van Krevelen R., Poelman R.: ‘A survey of augmented reality technologies, applications and limitations’, Int. J. Virtual Real., 2010, 9, p. 1
    1. DAQRI.: ‘Artoolkit.org: Open source augmented reality sdk’. [Online]. Available at: http://artoolkit.org/ [Accessed: 25-Feb-2018]