PhlatCam: Designed Phase-Mask Based Thin Lensless Camera

Abstract

We demonstrate a versatile thin lensless camera with a designed phase-mask placed at sub-2 mm from an imaging CMOS sensor. Using wave optics and phase retrieval methods, we present a general-purpose framework to create phase-masks that achieve desired sharp point-spread-functions (PSFs) for desired camera thicknesses. From a single 2D encoded measurement, we show the reconstruction of high-resolution 2D images, computational refocusing, and 3D imaging. This ability is made possible by our proposed high-performance contour-based PSF. The heuristic contour-based PSF is designed using concepts in signal processing to achieve maximal information transfer to a bit-depth limited sensor. Due to the efficient coding, we can use fast linear methods for high-quality image reconstructions and switch to iterative nonlinear methods for higher fidelity reconstructions and 3D imaging.

Fig. 1.

[Top] Non-lensing optics provides a way to achieve thin devices at low-cost. Among the various non-lensing optics, phase-masks are veratile in their designs and can produce a larger space of Point-spread-functions (PSF). [Bottom] PSFs from various optics are shown. Lensing optics have a small PSF support while non-lensing optics display large PSFs. The non-lensing optics’ PSFs were experimentally camptured.

Fig. 2.

Phase-masks are essentially transparent material with different heights at different locations. This causes phase modulation of incoming wavefront and resultant wave interference produces the PSF at the sensor plane. The above image shows the closeup image of the phase-mask using in PhlatCam. The right most image was taken using a scanning electron microscope (SEM).

Fig. 3.

Our proposed phase-mask framework takes the input of target PSF and the desired device geometry and outputs an optimized phase-mask design.

Fig. 4.. Conventional imaging and PhlatCam.

PhlatCam is 5–10× thinner and can reconstruct high-fidelity images from multiplexed measurements. Additionally, PhlatCam can function in more ways than conventional camera. Specifically PhlatCam can produce 2D images for any scene distance, refocused images at medium distance and 3D imaging at close distance.

Fig. 5.

Illustration of properties of phase mask in a lensless imaging setup.

Fig. 6.

Our Contour PSF is generated by applying canny edge detection on Perlin noise [42].

Fig. 7.. Modulation Transfer Function (MTF) of lensless point-spread-functions (PSFs).

The MTF is computed as the radially averaged magnitude spectrum of the PSFs. The PSFs compared are: Separable MSEQ [10], Fresnel zone apertures (FZA) [32], Tessellated spiral [13], Diffuser [14], Random binary, and our Contour PSF. The PSFs are visualized in Fig. 10. The magnitude spectrum of the proposed Contour PSF remains large for entire frequency range indicating better invertibility characteristic.

Fig. 8.

Visual illustration of the phase mask design.

Fig. 9.

The proposed PSF, designed phase-mask, and the experimentally realized PSF of PhlatCam are shown. The experimental PSF closely resembles the proposed PSF design, showing the effectiveness of the phase mask design framework.

Fig. 10.

Simulated reconstruction with previously proposed PSFs, random binary PSF and our Contour PSF Random binary PSF satisfies three of the four desired characteristics of PSF However, random binary PSF doesn’t satisfy the fourth characteristic, that is large contiguous regions of zero intensity. As seen from above, contour PSF consistently produces better results.

Fig. 11.

Experimental evaluation of our camera’s resolution using fluorescent USAF target. The inserts are shown for line pairs with contrast close to 20% or higher.

Fig. 12.. Experimental results: Photography.

The shortest distance to the scene is about 0.5 m, extending all the way to 3 m in the bottom scene. The bottom scene is a frame from video reconstruction. The video can be found in the supplementary material.

Fig. 13.. Experimental results: Microscopy.

[Top-left] Fluorescence microscopy setup. [Top-right] Ground truth image of fluorescent sample taken using 2.5× microscope objective lens. The sample is a root cell from lily-of-the-valley (Convallaria majalis) stained with green fluorescent dye. [Bottom] Sample capture (at 10 mm away) and reconstruction.

Fig. 14.

PhlatCam has depth dependent PSF that magnifies as the scene gets closer. The magnification falls with inverse depth relation. By looking at the correlation of PSFs, we can broadly categorize scene depth into 3 regimes. At close distances, the correlation falls quickly, enabling us to reconstruct 3D images. At the medium distances, the correlation falls gradually over a wider depth range. In this distance range, we can perform computational refocusing. At much larger depth, the dependence of PSF with depth saturates and all far scene points can be said to be beyond the hyperfocal distance of PhlatCam, thereby allowing only reconstruction of 2D images.

Fig. 15.

We showcase the refocusing ability of PhlatCam. Three objects at three different distances comes into focus when we use the appropriate depth PSF for the reconstruction.

Fig. 16.

We showcase the 3D image reconstruction ability of PhlatCam at very close distance. The scene is a handwritten text, written using phosphorescent paint. The letter ‘L’ is at the closest distance from the camera, at 10 mm, and the letter ‘T’ is at 38 mm from the camera. Hence, the scene ranges from 0 to 28 mm.

Fig. 17.. Experimental comparisons.

We experimentally compare results from three different prototypes — (a) amplitude mask designed for separable PSF (FlatCam [10]), (b) phase mask designed for separable PSF, and (c) proposed phase mask designed for Contour PSF. The camera thicknesses are approximately 2 mm. FlatCam reconstructions are performed using Tikhonov regularization [10], and using deep learning method [49]. Both the phase-mask reconstructions are performed with Eq. 9. The proposed PhlatCam produces cleaner and higher quality images.