Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Mar 9;118(10):e2024468118.
doi: 10.1073/pnas.2024468118.

Non-line-of-sight imaging over 1.43 km

Cheng Wu  1   2   3   4 , Jianjiang Liu  1   2   3   4   5 , Xin Huang  1   2   3   4 , Zheng-Ping Li  1   2   3   4 , Chao Yu  1   2   3   4 , Jun-Tian Ye  1   2   3   4 , Jun Zhang  1   2   3   4 , Qiang Zhang  1   2   3   4 , Xiankang Dou  1   2   5   6 , Vivek K Goyal  7 , Feihu Xu  8   2   3   4 , Jian-Wei Pan  8   2   3   4
Affiliations

Non-line-of-sight imaging over 1.43 km

Cheng Wu et al. Proc Natl Acad Sci U S A. .

Abstract

Non-line-of-sight (NLOS) imaging has the ability to reconstruct hidden objects from indirect light paths that scatter multiple times in the surrounding environment, which is of considerable interest in a wide range of applications. Whereas conventional imaging involves direct line-of-sight light transport to recover the visible objects, NLOS imaging aims to reconstruct the hidden objects from the indirect light paths that scatter multiple times, typically using the information encoded in the time-of-flight of scattered photons. Despite recent advances, NLOS imaging has remained at short-range realizations, limited by the heavy loss and the spatial mixing due to the multiple diffuse reflections. Here, both experimental and conceptual innovations yield hardware and software solutions to increase the standoff distance of NLOS imaging from meter to kilometer range, which is about three orders of magnitude longer than previous experiments. In hardware, we develop a high-efficiency, low-noise NLOS imaging system at near-infrared wavelength based on a dual-telescope confocal optical design. In software, we adopt a convex optimizer, equipped with a tailored spatial-temporal kernel expressed using three-dimensional matrix, to mitigate the effect of the spatial-temporal broadening over long standoffs. Together, these enable our demonstration of NLOS imaging and real-time tracking of hidden objects over a distance of 1.43 km. The results will open venues for the development of NLOS imaging techniques and relevant applications to real-world conditions.

Keywords: computational imaging; computer vision; non–line-of-sight imaging; optical imaging.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interest.

Figures

Fig. 1.
Fig. 1.
Long-range NLOS imaging experiment. (A) An aerial schematic of the NLOS imaging experiment over a (standoff distance of) 1.43-km free-space link, in which the setup is placed at A and the hidden scene is placed at B. (The geographic image is from Google Earth ©2020 Google.) (B) The optical setup of the NLOS imaging system, which consists of two synchronized telescopes for transmitter and receiver. A laser followed by galvo system and lenses are used for transmitting light pulses, while an InGaAs/InP SPAD together with a galvo system is used for collecting and detecting photons and recording their TOF information. (C) Schematic of the hidden scene in a room with a dimension size of 2 m×1 m. (D) An actual photograph of the NLOS imaging setup. (E and F) Zoomed-out and zoomed-in photographs of the hidden scene taken at location A, where only the visible wall can be seen. (G) Photograph of the hidden object, taken at the room located at B.
Fig. 2.
Fig. 2.
The reconstruction procedure of our approach. (A) The raw-data measurements are resampled along the time axis. (B) The spatiotemporal kernel is applied to the 3D raw data to solve the deconvolution with the SPIRAL3D solver. (C) Transforming the coordinate along depth from u to z to recover the hidden scene. (D) The final reconstructed result.
Fig. 3.
Fig. 3.
Comparison of the reconstructed results with different approaches. The NLOS measurements are computed by filtered back-projection with a 3D Laplacian of Gaussian filter (40), LCT with an iterative ADMM-based solver (22), and our proposed algorithm. (A) The reconstructed results for the hidden scene of mannequin. (B) The reconstructed results for the hidden scene of letter H.
Fig. 4.
Fig. 4.
NLOS real-time detecting and tracking experiment over 1.43 km. (A) The setup includes one telescope and a pulsed laser for transmitting light (L) and three telescopes and SPADs for detecting light (D1, D2, D3). (B) A representative raw measurement contains the first- and third-bounce photon counts. (C) Detection of the various positions for a single hidden object. (D) Detection of the positions for two simultaneous hidden objects. (E) The real-time tracking for a hidden moving object at a frame rate of 1 Hz. (F) The real-time tracking speed for a hidden moving object, where the retrieved speed is 1.28 cm/s, which matches the actual speed of 1.25 cm/s.
See this image and copyright information in PMC

References

    1. Altmann Y., et al. , Quantum-inspired computational imaging. Science 361, eaat2298 (2018). - PubMed
    1. Maeda T., Satat G., Swedish T., Sinha L., Raskar R., Recent advances in imaging around corners. arXiv:1910.05613 (12 October 2019).
    1. Faccio D., Velten A., Wetzstein G., Non-line-of-sight imaging. Nat. Rev. Phys. 2, 318–327 (2020).
    1. Kirmani A., Hutchison T., Davis J., Raskar R., “Looking around the corner using transient imaging” in Proceedings of the 12th IEEE International Conference on Computer Vision. (IEEE, 2009), pp. 159–166.
    1. Velten A., et al. , Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012). - PubMed

Publication types

LinkOut - more resources