Near-zero photon bioimaging by fusing deep learning and ultralow-light microscopy

Abstract

Enhancing the reliability and reproducibility of optical microscopy by reducing specimen irradiance continues to be an important biotechnology target. As irradiance levels are reduced, however, the particle nature of light is heightened, giving rise to Poisson noise, or photon sparsity that restricts only a few (0.5%) image pixels to comprise a photon. Photon sparsity can be addressed by collecting approximately 200 photons per pixel; this, however, requires long acquisitions and, as such, suboptimal imaging rates. Here, we introduce near-zero photon bioimaging, a method that operates at kHz rates and 10,000-fold lower irradiance than standard microscopy. To achieve this level of performance, we uniquely combined a judiciously designed epifluorescence microscope enabling ultralow background levels and AI that learns to reconstruct biological images from as low as 0.01 photons per pixel. We demonstrate that near-zero photon bioimaging captures the structure of multicellular and subcellular features with high fidelity, including features represented by nearly zero photons. Beyond optical microscopy, the near-zero photon bioimaging paradigm can be applied in remote sensing, covert applications, and biomedical imaging that utilize damaging or quantum light.

Keywords: imaging; microscopy; single-photon.

Fig. 1.

Ultralow-light imaging. Epifluorescence images of a Medicago truncatula root at increasing photon fluxes using conventional (A) and photon-counting (B) detectors. A 64% contrast at 266.3 photons (γ) per pixel is achieved using a conventional (sCMOS) detector (A), while photon-counting achieves 82% contrast at 41.3 photons per pixel (B). Denoising by discrete wavelet transforms (22) (Materials and Methods) of photon-counting images enables higher contrast (88%) at lower photon fluxes (1.02 photons per pixel) (C).

Fig. 2.

Near-zero photon imaging experiment and learning model. The near-zero photon epifluorescence setup (A) and U-Net architecture for image reconstruction, including the encoding and decoding paths linked with skip connections (B); each path consists of eight spatial tiers composed of convolutional blocks (CBs) of four repeating convolutional layers; each layer uses a LeakyReLU(α = 0.1) activation function followed by batch normalization; after each encoder CB, MaxPooling(2,2) downsampling is followed by a DropOut(0.5) layer to mitigate overfitting. (C) Photons per pixel for near-zero and high-flux images per biological sample and (D) the resulting Pearson’s coefficients between processed and unprocessed (raw) near-zero photon images with the ground truth; boxcharts and error bars represent 25 to 75%, and 10 to 90% ranges, respectively.

Fig. 3.

Near-zero photon imaging of fluorescently stained M. truncatula roots (A), Y. lipolytica cells highlighting their size (B) and lipid (TAG) content (C). All three cases display the raw (unprocessed, first column) and AI-reconstructed (second column) near-zero photon images, along with the respective high-flux ground truth images (third column). The fourth column compares exemplary 1D traces in the reconstructed near-zero photon and ground truth images that are represented by nearly zero photons in the raw image (each trace is noted in the respective images with dotted lines in μm units).

Fig. 4.

Near-zero phenotyping. Scatter plots comparing various phenotypic characteristics that were independently determined by near-zero (x-axis) and high-flux (y-axis) imaging. The phenotypic characteristics include the diameter (A) of primary M. truncatula roots, the size (B), and lipid (TAG) % content (C) of Y. lipolytica cells. Legends in all scatter plots detail the % error between near-zero and high-flux conditions (mean ± SEM). The photon fluxes (C–F) for the near-zero and high-flux ROIs used in phenotyping the root size (A), the size (B), and the lipid content of Y. lipolytica cells (C); boxcharts and error bars represent 25 to 75% and 10 to 90% ranges, respectively.

Fig. 5.

Generalizability and live-cell imaging. Time-lapse imaging of S. cerevisiae expressing RFP (A) at 0′, 12′, 43′, and 54′ time points, including the raw (first column) and reconstructed (second column) near-zero photon images, and a composite of the ground truth with brightfield images (third column); the Inset in the third column displays the brightfield images alone. Near-zero photon bioimaging accurately captures the cell shape and size, as well as their time-dependent positional shift that results from forces exerted by nearby growing cells. (B) Cell centroid, highlighted in red and blue in the reconstructed and ground truth images, respectively. (C) Photobleaching in Y. lipolytica stained with propidium iodide using near-zero photon bioimaging (i) and conventional epifluorescence (ii); blue line and red-shaded region represent the average and 95% CI for n = 22 single-cell observations for near-zero photon (i) and n = 48 for conventional (ii) imaging; Insets display examples of time-lapse series of Y. lipolytica cells at the noted timepoints (comparison performed at 1 kHz rates and 100× magnification).