Accurate localization microscopy by intrinsic aberration calibration

Abstract

A standard paradigm of localization microscopy involves extension from two to three dimensions by engineering information into emitter images, and approximation of errors resulting from the field dependence of optical aberrations. We invert this standard paradigm, introducing the concept of fully exploiting the latent information of intrinsic aberrations by comprehensive calibration of an ordinary microscope, enabling accurate localization of single emitters in three dimensions throughout an ultrawide and deep field. To complete the extraction of spatial information from microscale bodies ranging from imaging substrates to microsystem technologies, we introduce a synergistic concept of the rigid transformation of the positions of multiple emitters in three dimensions, improving precision, testing accuracy, and yielding measurements in six degrees of freedom. Our study illuminates the challenge of aberration effects in localization microscopy, redefines the challenge as an opportunity for accurate, precise, and complete localization, and elucidates the performance and reliability of a complex microelectromechanical system.

Fig. 1. Intrinsic aberrations enable accurate localization microscopy in six degrees of freedom.

a–c Fluorescence micrographs showing images of a particle at z positions of (a) 2 µm above, (b) near, and (c) 2 µm below best focus. The particle diameter is 1 μm and the resolution limit is 0.7 µm. Two aberration effects are apparent – symmetry variation from astigmatism and intensity variation from defocus. Dots indicate asymmetry in (a, c). Vertical positions correspond to white boxes in (d). d Schematic showing (red) fluorescent particles on part of a complex microsystem. We localize single particles in three dimensions and fit a rigid transformation to measure motion with six degrees of freedom – translations Δ_x, Δ_y, and Δ_z, intrinsic rotation γ about the axis of rotation $\overrightarrow{\mathit{u}}$, nutation β, and precession α. White arrows indicate play due to clearances in the microsystem. (d) Lateral dimensions are nearly to scale. Vertical dimensions are not to scale.

Fig. 2. Effects of intrinsic aberrations on apparent lateral position and particle image shape.

These data are from a representative calibration particle at a representative location in the imaging field. a Scatter plot showing apparent lateral position as a function of actual axial position. White data markers indicate the actual lateral position, which we define at the axial position of best focus z_f. Uncertainties are smaller than the data markers. b–f Plots showing the dependence on axial position of the parameters (b) ${\Delta }_x^{\prime }\left(z\right)=x^{\prime }\left(z\right)-x^{\prime }\left(z_f\right)$, (c) ${\Delta }_y^{\prime }\left(z\right)=y^{\prime }\left(z\right)-y^{\prime }\left(z_f\right)$, (d) A, (e) ρ, and (f) ${\rho }_A=\frac{\rho }{A_n}$, where $A_n=A/A_{\rho ={\rho }_0}$ is the amplitude after normalization to its value in the image for which ρ = ρ₀, with ρ₀ set to the minimum value of |ρ|. Fits of bivariate Gaussian models to emitter images determine the (black data markers) parameter values, and (green lines) polynomials model the z dependence for (b–c) lateral correction, (d) determination of the axial position of best focus z_f, and (e-f) axial localization. Residual values indicate an uncertainty for each parameter. Values in the bottom panels are uncertainties of (b-c) apparent lateral position ${\sigma }_{{\Delta }_x^{\prime }}$ and ${\sigma }_{{\Delta }_y^{\prime }}$ from the polynomial models, and (e–f) z position ${\sigma }_z$ from inversion of the polynomial models.

Fig. 3. Field dependence of local values of uncertainty components from axial localization.

a Line plots showing the relationship z(ρ_A) for many calibration particles. Three representative lines have colors corresponding to the map in (b) for local values of uncertainty ${\overline{\sigma }}_z$ from the polynomial models {z(ρ_A)}_cal. The overbar denotes the mean uncertainty over the axial range of (a). b Scatter plot showing the lateral positions of the calibration particles and corresponding values of ${\overline{\sigma }}_z$ for an axial range of 6 µm. c Histogram showing an asymmetric distribution of ${\overline{\sigma }}_z$ for an axial range of 6 µm. d Plot showing variation of the mean value of ${\overline{\sigma }}_z$ for all particles as a function of axial range for (triangles) {z(ρ)}_cal, (squares) {z(ρ_w)}_cal, and (circles) {z(ρ_A)}_cal. e Plot showing variation of the 68 % interpercentile range of ${\overline{\sigma }}_z$ for all particles as a function of axial range for (triangles) {z(ρ)}_cal, (squares) {z(ρ_w)}_cal, and (circles) {z(ρ_A)}_cal. Roundels in (d–e) correspond to (a–c). Uncertainties in (d–e) are smaller than the data markers.

Fig. 4. Field dependence of image shape and apparent lateral position.

a Schematic showing variation of image shape, with quantification by ρ_A, in three dimensions. b Scatter plots in perspective showing the lateral positions of the calibration particles and their values of ρ_A for the three representative values of z in (a). Black markers correspond to the particles in (f). c Surface plot showing a widefield calibration function of Zernike polynomials modeling variation in z for a representative value of ρ_A = 0. d–e Vector plots showing the apparent lateral motion of a subset of calibration particles for the three representative values of z in (a). f Grid of nine scatter plots showing the apparent lateral positions, through an axial range of 6 µm, of representative calibration particles from representative locations across the full lateral field. Black markers in the bottom plot of (b) show these representative locations. White data markers indicate the true lateral position of each particle, which we define as being at the z position of best focus, z_f, for each particle.

Fig. 5. Localization error throughout a deep and ultrawide field.

Gray data include calibration particles from the full square field and black data include only the particles within a subset circular field. a–c Scatter plots, histograms, and normal probability plots of the differences between local and widefield calibration functions for each calibration particle, which define the error of widefield calibration, at the z = 0 focal surface for (a) x, (b) y, and (c) z. The scatter plots show these errors as a function of the distance of each particle from the nominal center of the field. Histograms include a (white line) Gaussian model fit to the black data. d–f Plots showing root-mean-square (RMS) error as a function of z position for widefield calibration of (d) x, (e) y, and (f) z, using (triangle) nearest-neighbor interpolation, (square) natural-neighbor interpolation, and (circle) Zernike polynomials. Uncertainties in (d–f) are smaller than the data markers.

Fig. 6. Microsystem motion in three dimensions.

a Brightfield micrograph showing the drive motor, consisting of (i) a rotational actuator, (ii) a ring gear, and (iii) the load gear. b Brightfield micrograph magnifying the load gear and particles. The smaller dots with random spacing are florescent particles and the larger dots with regular spacing are etch holes. c Fluorescence micrograph showing a constellation of fluorescent particles on the surface of the load gear. The cross indicates the centroid of the subset of particles that we use for tracking. d Scatter plot showing the trajectory of the centroid of the particle constellation in three dimensions. Tilt is apparent. The position clusters in the x and y directions are due to the nature of the ratchet mechanism that rotates the load gear through 64 nominal orientations with each revolution. e Scatter plots showing centroid positions in the x-y plane for the nominal locations within the box in (d). Uncertainties for both lateral and axial positions are smaller than the data markers. a © 2020 IEEE. Reprinted, with permission, from Ref. .

Fig. 7. Play analysis.

Polar plot showing the range of motion of the constellation centroid in the radial direction and the axial direction at all 64 nominal orientations of the load gear. The bar length scales nonlinearly in the plane for clarity. The inset micrograph and black arrow show the direction of the coupling between the load gear and ring gear.

Fig. 8. Microsystem motion in six degrees of freedom.

a–c Plots and histograms showing (a) intrinsic rotations of the load gear in three-dimensional space γ, (b) the angle between the axis of rotation and the extrinsic z axis, or nutation β, and (c) the angle between the axis of rotation and the extrinsic x-z plane, or precession α. d Plot showing lines in the direction of (colors) the axis of rotation for each motion cycle, (gray) the mean axis of rotation, and (black) the extrinsic z axis. The cross denotes uncertainties. e–g Plots and histograms showing translation of the load gear in the (e) x, (f) y, and (g) z directions. h Plot showing the mean nutation $\overline{\beta }$ with a reciprocating rotation with each revolution of the load gear. Uncertainties are (a, e, f) smaller than data markers, (h) 68 % coverage intervals, and (b, c, g) as we describe in the Methods. a–c, e–g © 2020 IEEE. Reprinted, with permission, from Ref. .