System- and sample-agnostic isotropic three-dimensional microscopy by weakly physics-informed, domain-shift-resistant axial deblurring

Abstract

Three-dimensional subcellular imaging is essential for biomedical research, but the diffraction limit of optical microscopy compromises axial resolution, hindering accurate three-dimensional structural analysis. This challenge is particularly pronounced in label-free imaging of thick, heterogeneous tissues, where assumptions about data distribution (e.g. sparsity, label-specific distribution, and lateral-axial similarity) and system priors (e.g. independent and identically distributed noise and linear shift-invariant point-spread functions are often invalid. Here, we introduce SSAI-3D, a weakly physics-informed, domain-shift-resistant framework for robust isotropic three-dimensional imaging. SSAI-3D enables robust axial deblurring by generating a diverse, noise-resilient, sample-informed training dataset and sparsely fine-tuning a large pre-trained blind deblurring network. SSAI-3D is applied to label-free nonlinear imaging of living organoids, freshly excised human endometrium tissue, and mouse whisker pads, and further validated in publicly available ground-truth-paired experimental datasets of three-dimensional heterogeneous biological tissues with unknown blurring and noise across different microscopy systems.

Fig. 1. Principles of SSAI-3D.

a SSAI-3D enables robust isotropic resolution recovery across diverse 3D imaging systems (confocal, light-sheet, wide-field, and nonlinear) and diverse 3D biological samples (organelles, cells, tissues, and organs). Created in BioRender. Liu, K. (2024) BioRender.com/f26y695. b The deblurring network is initialized with a large pre-trained network for blind deconvolution on a large dataset of natural image pairs. Example images were sourced from the REDS dataset, released under a CC BY 4.0 license, which permits commercial use with proper attribution. c Starting with a single microscopy-specific image stack where the axial resolution is worse than the lateral resolution, lateral images are blurred with a series of PSFs of different sizes and orientations to generate a self-supervised dataset. Then, generating zero-shot metrics using ~1% of this dataset, a surgeon network is employed to select the critical layers to fine-tune in the large pre-trained deblurring network. Only ~10% layers are selected and sparsely fine-tuned according to the generated self-supervised dataset. Given unseen axial images, the fine-tuned deblurring network predicts deblurred images with isotropic resolution. Insets in the images represent the corresponding Fourier spectrums.

Fig. 2. Performance of SSAI-3D on synthetic samples.

Simulation of bead (a) and strand structures (b) reveals the performance of SSAI-3D and existing methods (c, d). The simulation objects are visualized using depth-coded projections with the colorbar representing the actual depth. Reconstructed axial resolution (mean ± standard deviation, n = 50 beads) is labeled in c. e, f Characterization of reconstruction fidelity for beads (a) and strands (b). Box bounds represent the upper and lower quartiles, lines within boxes indicate medians, and whiskers extend to data points within 1.5 times the interquartile range (IQR), with outliers plotted individually beyond this range (n = 100 slices). g, h Ablation study on the performance of different levels of blurring for beads (a) and strands (b). i, j Robustness of reconstruction against PSF mismatch for beads (a) and strands (b). A blurring with a standard deviation of 5 is assumed to be known for CARE. k Comparison of the training time. Scale bars: 6 µm (a, c); 60 µm (b, d). Source data are provided as a Source Data file.

Fig. 3. Effect of lateral-axial similarity of 3D data using simulations and tissues.

Simulation of sphere (high lateral-axial similarity, a) and cylinder (low lateral-axial similarity, d) reveals the performance of SSAI-3D and existing methods (c, f). b, e Relationship between training MSE on lateral images and inference MSE on axial images indicates robustness of different methods against distribution shifts. In label-free nonlinear imaging, biological tissues exhibit different levels of lateral-axial similarity (high in living and intact human blood-brain barrier microfluidic model (g) and low in freshly excised human endometrial tissue (i)). h, j Raw lateral and axial images of g and i, as well as restored axial images using SSAI-3D. Arrows: symmetric endothelial cells (h) and polarized epithelial glands (j). Scale bars: 50 µm. Source data are provided as a Source Data file.

Fig. 4. Effect of optical aberrations and noise on deblurring performance using simulations.

a Example of optical aberrations in real microscopy system (spatially-varying PSF in mesoSPIM). b Example of noise in real microscopy system (low-light condition in dSLAM). c Ground truth image for simulation on effect of optical aberrations and noise. Four different levels of optical aberrations and noise with the corresponding raw images (d Shift-invariant PSF (ideal case); e Shift-variant PSF; f Shift-variant PSF with rotations along z-direction; g Shift-variant PSF with rotations along z-direction and noise across the entire imaging volume). h–k Comparison of Self-Net, CARE, and SSAI-3D on deblurring performance using SSIM in scenarios d–g. Bottom: Blow-ups of the co-registered images within the white box. Scale bars: 2 mm (a); 25 µm (b–k).

Fig. 5. SSAI-3D allows isotropic resolution recovery across different imaging systems and samples.

Each example includes the raw and restored images (left), along with their respective blow-ups (highlighted in the white boxes, middle) and Fourier spectrums (right, labeled numbers represent the lateral and axial resolutions). a Cleared and stained mouse brain vasculature from light-sheet microscopy. b Freshly excised human endometrial tissue from multiphoton autofluorescence microscopy. c Cleared Thy1-GFP mouse brain neurons from confocal microscopy. d Fixed mouse whisker follicles from second harmonic generation (SHG) microscopy. e Cleared mouse liver from wide-field microscopy. f Freshly excised human endometrial tissue from third harmonic generation (THG) microscopy. Scale bars: 20 µm.

Fig. 6. Validated isotropic resolution recovery of SSAI-3D facilitates downstream biological analysis.

a Comparison of software-corrected (SSAI-3D) and hardware-corrected (mesoSPIM) axial image of mouse brain from light-sheet microscopy. b Neuron detection accuracy using raw, software-corrected, and hardware-corrected images. Arrows in a and b are pointing to a same neuron. c Comparison of software-corrected (SSAI-3D) and hardware-corrected (multiview confocal) axial image of mitochondria from confocal microscopy. d Statistics of the volume of DNA puncta in mitochondria. Inset: overall statistics of raw data. Arrows in c represent the same DNA puncta with the volume marked in d. Scale bars: 1 mm (a, b); 1 µm (c, d). Source data are provided as a Source Data file.