Metasurface-driven full-space structured light for three-dimensional imaging

Abstract

Structured light (SL)-based depth-sensing technology illuminates the objects with an array of dots, and backscattered light is monitored to extract three-dimensional information. Conventionally, diffractive optical elements have been used to form laser dot array, however, the field-of-view (FOV) and diffraction efficiency are limited due to their micron-scale pixel size. Here, we propose a metasurface-enhanced SL-based depth-sensing platform that scatters high-density ~10 K dot array over the 180° FOV by manipulating light at subwavelength-scale. As a proof-of-concept, we place face masks one on the beam axis and the other 50° apart from axis within distance of 1 m and estimate the depth information using a stereo matching algorithm. Furthermore, we demonstrate the replication of the metasurface using the nanoparticle-embedded-resin (nano-PER) imprinting method which enables high-throughput manufacturing of the metasurfaces on any arbitrary substrates. Such a full-space diffractive metasurface may afford ultra-compact depth perception platform for face recognition and automotive robot vision applications.

Fig. 1. Metasurface-based SL imaging platform scattering high-density diffracted beams into the full 180° FOV.

Under the illumination of a polarization-independent coherent laser source, the proposed metasurfaces generate ~10 K points over the entire 180° space. The depths of the dot arrays illuminated on the objects are extracted using a stereo matching algorithm.

Fig. 2. Design principle and experimental demonstration of full-space diffractive metasurface.

a–g Flowchart of designing the metasurfaces. The color of the border, blue and red, represents the space and frequency domain, respectively. a Desired spatial frequency distribution. All the propagating waves are designated to have uniform amplitude, excluding excited evanescent waves from the target distribution. b The kernel function of the convolution. From the target spatial frequency distribution, the phase-only plate, ranging from 0 to 2$\pi$, is retrieved from an iterative Fourier transform algorithm, namely the GS algorithm. c 2D Dirac comb function at the space domain representing the supercell arrangement. d Fourier transformed of 2D Dirac comb function with period of 1/nP. e The building blocks of the geometric phase-based metasurfaces. Simulated CE of the meta-atom using RCWA as a function of length (L) and width (W) with fixed height (H) and pitch (P). f Final phase $\phi$ obtained from convolution of kernel function and 2D Dirac comb function at the space domain. g The final diffracted pattern representing discrete order of diffraction, which is same with multiplication of initial target (a) and 2D Dirac comb at the frequency domain (d) according to the convolution theorem. h Experimental demonstration of the full-space diffracting metasurfaces at the observation plane placed front and side of the metasurface with a scanning electron microscopy (SEM) image of the fabricated metasurfaces.

Fig. 3. Comparison between simulated and measured diffraction patterns of 1D diffractive metasurfaces.

a–c Simulated distribution of amplitude of electric field at the plane normal of the metasurface (a) together with magnifications of the far (b), and near fields (c). d Schematic of the optical setup to measure power of each point by rotating power meter at the operation wavelength of 633 nm where the incident laser light is chirped to fit into the size of metasurfaces. SEM image of the corresponding metasurfaces. e Plot of number of diffraction orders, and relative intensities of simulated and measured diffracted beam with respect to propagating angle. The intensities are plotted with averaged value with error bars, as propagating angle is grouped at intervals of 10°.

Fig. 4. Depth estimation of 3D objects.

a Schematic of the optical setup for depth estimation using 2D dot arrays scattered on objects. b Schematic of the stereo matching algorithm used in (a). The algorithm uses camera trigonometry and the coordinate difference of the dot points in two image planes. c A photograph of the objects; one placed normally and the other placed at 50° with respect to laser beam axis. d 3D depth reconstruction results of objects in (c) in different rotation view. The first and second row are the depth images of object 1 and object 2, respectively. The depths are presented in colors with respect to the image plane of camera described as the red arrows.

Fig. 5. Demonstration of prototype device by direct printing of nano-PER on curved surface of glasses.

a The flow chart of nano-PER based imprinting fabrication process. b–d SEM image of the master stamp fabricated by EBL (tilted view) (b), soft mold composed of h-PDMS and PDMS (c), and replicated nano-PER-based metasurface (tilted view) (d). e Replicated metasurface on the curved surface of safety glasses. The size of printed metasurface is 510 μm by 510 μm. f Experimental demonstration of full-space covering SL of 2D line array. Laser light illuminate the nano-PER-based metasurface printed on surface of glasses exhibiting application for ultra-compact depth sensor for AR glasses.