Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Aug 1;18(8):2505.
doi: 10.3390/s18082505.

An Augmented Reality System Using Improved-Iterative Closest Point Algorithm for On-Patient Medical Image Visualization

Ming-Long Wu  1 , Jong-Chih Chien  2 , Chieh-Tsai Wu  3 , Jiann-Der Lee  4   5   6
Affiliations

An Augmented Reality System Using Improved-Iterative Closest Point Algorithm for On-Patient Medical Image Visualization

Ming-Long Wu et al. Sensors (Basel). .

Abstract

In many surgery assistance systems, cumbersome equipment or complicated algorithms are often introduced to build the whole system. To build a system without cumbersome equipment or complicated algorithms, and to provide physicians the ability to observe the location of the lesion in the course of surgery, an augmented reality approach using an improved alignment method to image-guided surgery (IGS) is proposed. The system uses RGB-Depth sensor in conjunction with the Point Cloud Library (PCL) to build and establish the patient's head surface information, and, through the use of the improved alignment algorithm proposed in this study, the preoperative medical imaging information obtained can be placed in the same world-coordinates system as the patient's head surface information. The traditional alignment method, Iterative Closest Point (ICP), has the disadvantage that an ill-chosen starting position will result only in a locally optimal solution. The proposed improved para-alignment algorithm, named improved-ICP (I-ICP), uses a stochastic perturbation technique to escape from locally optimal solutions and reach the globally optimal solution. After the alignment, the results will be merged and displayed using Microsoft's HoloLens Head-Mounted Display (HMD), and allows the surgeon to view the patient's head at the same time as the patient's medical images. In this study, experiments were performed using spatial reference points with known positions. The experimental results show that the proposed improved alignment algorithm has errors bounded within 3 mm, which is highly accurate.

Keywords: head-mounted display; image alignment; improved-ICP algorithm.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
(a) A dummy head model; and (b) model of intracranial vascular tissue.
Figure 2
Figure 2
The flowchart of I-ICP alignment algorithm.
Figure 3
Figure 3
ICP Trapped in Locally Optimal Solution.
Figure 4
Figure 4
Example Illustration of the RMS Solution.
Figure 5
Figure 5
A self-made Vuforia marker.
Figure 6
Figure 6
The process of conversion between different coordinate systems.
Figure 7
Figure 7
Stanford Bunny alignment results: (a) initial positions; (b) I-ICP; (c) PCL-ICP; (d) Go-ICP; (e) CPD-ICP (rigid); (f) CPD-ICP (affine); and (g) CC-ICP.
Figure 8
Figure 8
Compare the TRE of different ICP methods.
Figure 9
Figure 9
The Result of Marker Detection and Overlaying the Medical Images over the Dummy Head.
Figure 10
Figure 10
Display of cursor for interactive rotation, and scaling.
Figure 11
Figure 11
Diagram of the computation of coordinate conversion validation errors.
Figure 12
Figure 12
Comparison of the TREs for different ICP methods.
Figure 13
Figure 13
Alignment result of using I-ICP on Dummy Head #1.
See this image and copyright information in PMC

References

    1. Kundu S.N., Muhammad N., Sattar F. Using the augmented reality sandbox for advanced learning in geoscience education; Proceedings of the 2017 IEEE 6th International Conference on Teaching, Assessment, and Learning for Engineering (TALE); Hong Kong, China. 12–14 December 2018; pp. 13–17.
    1. Ashfaq Q., Sirshar M. Emerging trends in augmented reality games. Computing, Mathematics and Engineering Technologies; Proceedings of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET); Sukkur, Pakistan. 3–4 March 2018; pp. 1–7.
    1. Hamacher A., Kim S.J., Cho S.T., Pardeshi S., Lee S.H., Eun S.J., Whangbo T.K. Application of Virtual, Augmented, and Mixed Reality to Urology. Int. Neurourol. J. 2016;20:172–181. doi: 10.5213/inj.1632714.357. - DOI - PMC - PubMed
    1. Virtual Reality. [(accessed on 16 February 2018)]; Available online: https://www.wareable.com/vr/how-does-vr-work-explained.
    1. Traub J., Sielhorst T., Heining S., Navab N. Advanced Display and Visualization Concepts for Image Guided Surgery. J. Disp. Technol. 2008;4:483–490. doi: 10.1109/JDT.2008.2006510. - DOI