Camera array based light field microscopy

Abstract

This paper proposes a novel approach for high-resolution light field microscopy imaging by using a camera array. In this approach, we apply a two-stage relay system for expanding the aperture plane of the microscope into the size of an imaging lens array, and utilize a sensor array for acquiring different sub-apertures images formed by corresponding imaging lenses. By combining the rectified and synchronized images from 5 × 5 viewpoints with our prototype system, we successfully recovered color light field videos for various fast-moving microscopic specimens with a spatial resolution of 0.79 megapixels at 30 frames per second, corresponding to an unprecedented data throughput of 562.5 MB/s for light field microscopy. We also demonstrated the use of the reported platform for different applications, including post-capture refocusing, phase reconstruction, 3D imaging, and optical metrology.

Keywords: (100.3010) Image reconstruction techniques; (110.1758) Computational imaging; (170.0110) Imaging systems; (180.6900) Three-dimensional microscopy.

Fig. 1

Schematic of the proposed CALM system (top) and a photograph showing the prototype system (bottom) ( Visualization 1). We employed 25 cameras to capture the color light field videos of the microscopic specimens with a spatial resolution of 0.79 megapixels at 30 frames per second. The light microscope (Olympus IX73) is configured with a objective lens with 10× magnification, 0.4 numerical aperture and 2.65mm field-of-view.

Fig. 2

Evaluating resolution and depth of field of the prototype system. The captured light fields have large depth of field (first row), and can achieve an optical sectioning comparable to conventional microscopy (third row) by combining 25 views for refocusing (second row). The axial resolution of our system is 8.8μm and the resolvable group of stripe for each view is 5.6 (line width 8.77μm).

Fig. 3

Capturing the light field of rose petal (left, Visualization 2) and integrated circuit board (right, Visualization 3) for synthetic refocusing and recovering its 3D structure. The parallaxes of specimens for different views are successfully recorded (first and third column), which can be used for post-capture refocusing (second and fourth column, top) and 3D reconstruction (second and fourth column, bottom).

Fig. 4

The applications of light field video of drosophila larva to 3D DPC and phase reconstruction ( Visualization 4). With the captured light field video of a living drosophila larva (first row), the 3D differential phase-contrast (DPC) video pair is computed (second and third rows) for recovering the phase video (forth row) which represents sample’s optical path length (e.g. height).

Fig. 5

The light field and synthetic refocusing videos of the Caenorhabditis elegans (C. elegans) in the water ( Visualization 5). We captured the light field video of large scale C. elegans movement in the water (first row), and the synthetic refocusing video from it reveals the relative spatial locations of different C. elegans along the axial dimension (second row).

Fig. 6

Quantitative phase reconstruction for microlens array with the captured light field ( Visualization 6). In order to quantitatively evaluate the accuracy of captured light fields by using our approach, we reconstructed the phase of a microlens array (top-right) with its snapshot light field (top-left) and compared the recovered shape with a scanning confocal light microscopy result (bottom).

Fig. 7

Synthetic aperture and refocusing from the captured light field of cotton ( Visualization 7). The aliasing is occurred in the refocusing and synthetic aperture results in this example due to the large-depth-range and thin-structure of the sample. We used all perspective images of the capture light field (first and second columns) for refocusing (third column) and adopted center 3 × 3 views for synthetic aperture (forth column).