Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 May:2022:5587-5593.
doi: 10.1109/icra46639.2022.9812257. Epub 2022 Jul 12.

SAGE: SLAM with Appearance and Geometry Prior for Endoscopy

Xingtong Liu  1 , Zhaoshuo Li  1 , Masaru Ishii  2 , Gregory D Hager  1 , Russell H Taylor  1 , Mathias Unberath  1
Affiliations

SAGE: SLAM with Appearance and Geometry Prior for Endoscopy

Xingtong Liu et al. IEEE Int Conf Robot Autom. 2022 May.

Abstract

In endoscopy, many applications (e.g., surgical navigation) would benefit from a real-time method that can simultaneously track the endoscope and reconstruct the dense 3D geometry of the observed anatomy from a monocular endoscopic video. To this end, we develop a Simultaneous Localization and Mapping system by combining the learning-based appearance and optimizable geometry priors and factor graph optimization. The appearance and geometry priors are explicitly learned in an end-to-end differentiable training pipeline to master the task of pair-wise image alignment, one of the core components of the SLAM system. In our experiments, the proposed SLAM system is shown to robustly handle the challenges of texture scarceness and illumination variation that are commonly seen in endoscopy. The system generalizes well to unseen endoscopes and subjects and performs favorably compared with a state-of-the-art feature-based SLAM system. The code repository is available at https://github.com/lppllppl920/SAGE-SLAM.git.

PubMed Disclaimer

Conflict of interest statement

Under a license agreement between Galen Robotics, Inc and JHU, Dr. Taylor and JHU are entitled to royalty distributions on technology related to this publication. Dr. Taylor also is a paid consultant to and owns equity in Galen Robotics, Inc. This arrangement has been reviewed and approved by JHU in accordance with its conflict-of-interest policies.

Figures

Fig. 1:
Fig. 1:. Overall diagram of SLAM system.
The top left shows the module relationship in our SLAM system. The top right demonstrates the network prediction and pose-scale optimization within the Camera Tracking module. The bottom left shows the process of global loop detection and closure within the Loop Closure module. The bottom right demonstrates the optimization in the Mapping module, where pair-wise factors between non-adjacent keyframes are not shown. More details are described in Sec. IV.
See this image and copyright information in PMC

References

    1. De Groen PC, “History of the endoscope [scanning our past],” vol. 105, no. 10, pp. 1987–1995, 2017.
    1. Fockens P, “Endoscopic management of perforations in the gastrointestinal tract,” Gastroenterol. Hepatol, vol. 12, no. 10, pp. 641–643, Oct. 2016. - PMC - PubMed
    1. Eliashar R, Sichel J-Y, Gross M, Hocwald E, Dano I, Biron A, Ben-Yaacov A, Goldfarb A, and Elidan J, “Image guided navigation system-a new technology for complex endoscopic endonasal surgery,” Postgrad. Med. J, vol. 79, no. 938, pp. 686–690, Dec. 2003. - PMC - PubMed
    1. Grasa OG, Bernal E, Casado S, Gil I, and Montiel J, “Visual slam for handheld monocular endoscope,” TMI, vol. 33, no. 1, pp. 135–146, 2013. - PubMed
    1. Mahmoud N, Hostettler A, Collins T, Soler L, Doignon C, and Montiel J, “Slam based quasi dense reconstruction for minimally invasive surgery scenes,” arXiv, 2017.

LinkOut - more resources