Real-time cryo-electron microscopy data preprocessing with Warp

Abstract

The acquisition of cryo-electron microscopy (cryo-EM) data from biological specimens must be tightly coupled to data preprocessing to ensure the best data quality and microscope usage. Here we describe Warp, a software that automates all preprocessing steps of cryo-EM data acquisition and enables real-time evaluation. Warp corrects micrographs for global and local motion, estimates the local defocus and monitors key parameters for each recorded micrograph or tomographic tilt series in real time. The software further includes deep-learning-based models for accurate particle picking and image denoising. The output from Warp can be fed into established programs for particle classification and 3D-map refinement. Our benchmarks show improvement in the nominal resolution, which went from 3.9 Å to 3.2 Å, of a published cryo-EM data set for influenza virus hemagglutinin. Warp is easy to install from http://github.com/cramerlab/warp and computationally inexpensive, and has an intuitive, streamlined user interface.

Figure 1. Warp handles all pre-processing steps to close a gap in the 2D cryo–EM pipeline.

Data is acquired by the microscope in an automated fashion and stored as compressed stacks of movie frames. Warp continuously monitors its input folder for new files, and subjects them to all steps of the pre-processing pipeline: frame alignment, CTF estimation and particle picking. Warp writes out a stack of particles for each pre-processed micrograph and maintains a dynamically updated STAR file with references to all particles and their local CTF parameters. This file can be used as a data source in a tool such as cryoSPARC, which will periodically run subsequent processing steps like 2D classification and ab initio reconstruction on the latest set of particles.

Figure 2. Automated particle picking with Warp’s deep learning-based BoxNet

Representative example of automated particle picking with BoxNet in Warp on a micrograph with high-contrast artifacts. Areas masked out automatically by BoxNet are colored purple. The generic version of BoxNet was never presented with the sample during training. The re-trained version was given 5 micrographs of the same sample, which did not include the one shown. The template-based picking in RELION used 25 class averages derived from 3000 particles, filtered to 20 A. RELION’s results show the 120 highest-scoring positions. For visualization purposes, the micrograph was deconvolved, high-pass filtered and cropped at the borders.

Figure 3. Warp's 2D pipeline improves cryo-EM density for influenza hemagglutinin.

As a benchmarking case we used the published EMPIAR-10097 data set containing influenza hemagglutinin trimer particles. 'Original set': 130,000 pre-extracted particles from EMPIAR-10097 with their original CTF parameters; 'Original set, best class': 57,346 particles from 'Original set' after 3D classification in cryoSPARC with their original CTF parameters; 'Original set, best class + Warp CTF': the same 57 346 particles, but with Warp's CTF estimates; 'Full Warp pipeline': 249,495 particles obtained from the raw EMPIAR-10097 data after unsupervised pre-processing in Warp and 3D classification in cryoSPARC. (a) Isosurface renderings of the 3D maps generated with cryoSPARC using the respective sets of particles and CTF parameters, filtered to local resolution using the auto-tightened masks generated by cryoSPARC. (b) Global masked FSC plots, and histograms of the local resolution used to filter the maps depicted in (a). 'EMPIAR-10097 half-maps' refers to the original half-map volumes deposited in EMPIAR-10097, obtained from the same 130,000 particles as 'Original set'. (c) Histogram comparison between the original defocus parameters and those estimated by Warp for the 130 000 particles from 'Original set'. The mean value for each metric is specified underneath the horizontal axis in the same color as the corresponding histogram.

Figure 4. Warp’s 2D pipeline in combination with RELION 3.0 improves cryo-EM density for β-galactosidase.

For our second benchmark, we used the published EMPIAR-10061 data set containing β-galactosidase particles. The data were processed using the full Warp pre-processing pipeline, and beam tilt, per-particle defocus and frame alignment were later refined against high-resolution references in RELION 3.0 to assess the additional improvement provided by these refinements. (a) Isosurface rendering of the 1.86 Å map (left) and a detailed view of some of its sidechains, clearly displaying the aromatic rings (right). (b) Global masked FSC plots for the map obtained with the Warp pipeline only, and for the additive effects of reference-based beam tilt and per-particle defocus refinement, as well as particle polishing in RELION 3.0. (c) Average CTF fit quality curves for aligned movie averages produced with MotionCor2 and Warp. Warp’s averages can be fitted to a higher resolution, indicating more accurate frame alignment. For comparison, a fit quality curve is also included for amplitude spectra obtained from the average of individual frame spectra, which are invariant to residual inter-frame motion and radiation damage.

Figure 5. Effect of using the full local 3D CTF for template matching in tomograms.

During template matching, Warp multiplies the rotated 3D reference by the local 3D CTF before correlating it to a local portion of the tomogram volume, as opposed to multiplying it by a binary missing wedge mask. This produces sharper correlation peaks. (a) XY slice through a tomogram reconstructed from EMPIAR-10045 data. The faint shapes of 80S ribosomes are visible. (b) XY slice through the correlation volume at the same location as (a), using a binary missing wedge mask. White indicates higher correlation. The peaks are broad and hard to distinguish against the background. (c) XY slice through the correlation volume at the same location as (a), using the full local 3D CTF. White indicates higher correlation. The peaks are sharper, leading to higher template matching accuracy. (d) Rotational average of a 48 px window around all correlation peaks, mean-subtracted and normalized against the respective correlation background. 3D CTF-aware template matching (+CTF) produces peaks rising 2.7 times higher above the background compared to binary missing wedge masks (–CTF).

Figure 6. Sub-tomogram averaging results obtained by using Warp’s tilt series CTF estimation and sub-tomogram export.

To assess the benefits of the proposed CTF estimation and sub-tomogram export strategies, data from EMPIAR-10045 and EMPIAR-10164 were pre-processed and exported in Warp, and refined in RELION 3.0. Improved resolution was observed in both cases compared to published results. (a) Isosurface rendering (left) and FSC plot (right) of the 80S ribosome sub-tomogram average obtained from EMPIAR-10045 data. The originally published resolution was 12.8 Å. (b) Isosurface rendering (left) and FSC plot (right) of the HIV-1 sub-tomogram average obtained from 12% of the EMPIAR-10164 data. The originally published resolution for this subset was 3.9 Å.