Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction

Abstract

Super-resolved structured illumination microscopy (SR-SIM) is among the fastest fluorescence microscopy techniques capable of surpassing the optical diffraction limit. Current custom-build instruments are able to deliver two-fold resolution enhancement with high acquisition speed. SR-SIM is usually a two-step process, with raw-data acquisition and subsequent, time-consuming post-processing for image reconstruction. In contrast, wide-field and (multi-spot) confocal techniques produce high-resolution images instantly. Such immediacy is also possible with SR-SIM, by tight integration of a video-rate capable SIM with fast reconstruction software. Here we present instant SR-SIM by VIGOR (Video-rate Immediate GPU-accelerated Open-Source Reconstruction). We demonstrate multi-color SR-SIM at video frame-rates, with less than 250 ms delay between measurement and reconstructed image display. This is achieved by modifying and extending high-speed SR-SIM image acquisition with a new, GPU-enhanced, network-enabled image-reconstruction software. We demonstrate high-speed surveying of biological samples in multiple colors and live imaging of moving mitochondria as an example of intracellular dynamics.

Fig. 1

Schematic overview of the setup. a Opto-mechanics are based on ref. , modified for achromatic excitation and multicolor detection, for details see Supplementary Fig. 8. b The control flow diagram shows system components receiving both real-time timing pulses (TTL) and network-based control commands, for details see Supplementary Fig. 4. c The data flow diagram details how raw data acquired by the cameras travels through the on-the-fly reconstruction pipeline, being cached in ring-buffers for robustness and optionally saved for later, offline analysis. See Supplementary Fig. 3 for more details

Fig. 2

Timing diagram and overview of achievable frame rates. a The diagram shows the intricate timing of all components (SLM, cameras, AOTF/Laser) during a high-speed SIM acquisition for two colors, with 1 ms exposure time and a 512 × 512 pixel ROI. The cameras have to be set to 520 × 520, as explained in Supplementary Note 1. Illuminating the sample in one color channel while the camera of the other color channel is reading out its sensor gives a speedup over an image-splitter approach with only one camera. More detailed diagrams are shown in Supplementary Fig. 1 (two-color) and Supplementary Fig. 2 (three-color). b The table shows the highest achievable frame rates for a selection of different combinations of imaging area, number of simultaneously imaged color channels and exposure times (rounded down to integers)

Fig. 3

Demonstration of high-speed live-cell imaging and instant image reconstruction. A typical survey of stained U2OS cells was carried out with our SIM system. Video-recording (full version in Supplementary Movie 1) of the real-time results as presented to the user (column a), and the SIM reconstructions of the 488 nm channel (column b), stained for mitochondria (MitoTracker Green), and of the 647 nm channel (column c), stained for the endoplasmic reticulum (ER-Tracker Red). Column d shows an overlay of both reconstructed channels. Imaging was performed in dual color at 10.4 fps with 2 ms illumination time per raw frame (see Fig. 2b), multiple cells were surveyed in quick succession (see timestamp in each row). SIM allows to clearly resolve mitochondria and the endoplasmic reticulum

Fig. 4

Demonstration of the high image frame rates achieved by high-speed SR-SIM. a Cutouts of eight consecutive SR-SIM frames taken at 57.8 SIM frames (520 raw frames) per second, with 0.5 ms illumination time for each of the nine raw frames per SR-SIM image. The sample consists of 0.2 µm TetraSpeck Microspheres in a 50/50 mixture of stock solution and glycerol. The excitation wavelength was 647 nm, laser power was about 3 mW before entering the objective lens. The short acquisition time allows the acquisition of images with low motion blur, even though the sample is diffusing quickly, and the high SR-SIM frame rate leads to a high temporal resolution. b Movement of the diffusing microspheres along the x-axis over time displayed as a kymograph. It spans over 107 consecutive SIM frames, or ~1.85 s, including the images shown in (a). Note that the diffusing microspheres could also be observed in real time and super-resolution by the system operator. Fixed microspheres were also imaged to demonstrate the general capabilities of the SIM system, see Supplementary Fig. 7

Fig. 5

Demonstration of the system in use for live-cell imaging of mitochondrial dynamics. Single-color imaging of U2OS cells with 5 ms raw frame exposure time (see Fig. 2b), with overview (a) and time series of cut-out (b). The dynamics could be observed in real time and super-resolution by the system operator. The microscope hardware could easily provide higher speeds and continuous data acquisition, however, current dyes are limited in both brightness and photo-stability. Nonetheless, short exposure times are still very helpful in time-lapsed acquisition, as they suppress motion blur

Fig. 6

Demonstration of three-color live-cell imaging. These images were acquired with 10 ms raw frame exposure time and 2.4 (time lapsed) multicolor SIM frames per second. Live U2OS cells were stained for nucleic acids with SYTO 9, mitochondria and endoplasmic reticulum with MitoTracker Red, and polymerized tubulin filaments with Tubulin Tracker Deep Red. The individual channels and an overlay of all three channels (SYTO 9 in cyan, MitoTracker Red in green, and Tubulin Tracker Deep Red in magenta) are shown as reconstructed Wiener filtered wide-field and FairSIM images, respectively. The scale bar in the overlay images represents 5 µm