Real-time multi-view deconvolution

Abstract

In light-sheet microscopy, overall image content and resolution are improved by acquiring and fusing multiple views of the sample from different directions. State-of-the-art multi-view (MV) deconvolution simultaneously fuses and deconvolves the images in 3D, but processing takes a multiple of the acquisition time and constitutes the bottleneck in the imaging pipeline. Here, we show that MV deconvolution in 3D can finally be achieved in real-time by processing cross-sectional planes individually on the massively parallel architecture of a graphics processing unit (GPU). Our approximation is valid in the typical case where the rotation axis lies in the imaging plane.

Availability and implementation: Source code and binaries are available on github (https://github.com/bene51/), native code under the repository 'gpu_deconvolution', Java wrappers implementing Fiji plugins under 'SPIM_Reconstruction_Cuda'.

Contact: bschmid@mpi-cbg.de or huisken@mpi-cbg.de

Supplementary information: Supplementary data are available at Bioinformatics online.

Fig. 1.

Plane-wise multi-view deconvolution concept and performance. (a) Concept of plane-wise deconvolution for two views. Each dataset is resliced into planes orthogonal to the microscope’s rotation axis. Datasets are deconvolved plane-by-plane. (b) Memory requirements for traditional 3D and our plane-wise multi-view deconvolution, for various data sizes and numbers of views, on a logarithmic scale. (c) Execution times for plane-wise multi-view deconvolution, implemented on GPU and CPU, and 3D deconvolution, with and without GPU support. Memory requirements for 3D deconvolution timings for the 2048³ pixel dataset were beyond the capabilities of our workstation. (d–i) Resulting images of a 9 h post-fertilization transgenic Tg(h2afva:h2afva-mCherry) zebrafish embryo, using different methods (view along the rotational axis, scale bar 100 $\mu \text{m}$, 10 $\mu \text{m}$ in the inset): (d, e) acquired raw data, (f–i) fusion performed by (f) averaging, (g) entropy-weighted averaging, (h) 3D multi-view deconvolution and (i) plane-wise multi-view deconvolution (10 iterations). (Dell T6100, Intel E5-2630 @2.3 GHz 2 processors, 64 GB RAM; Graphics card: Nvidia GeForce GTX TITAN Black)