Stitching and registering highly multiplexed whole-slide images of tissues and tumors using ASHLAR

Abstract

Motivation: Stitching microscope images into a mosaic is an essential step in the analysis and visualization of large biological specimens, particularly human and animal tissues. Recent approaches to highly multiplexed imaging generate high-plex data from sequential rounds of lower-plex imaging. These multiplexed imaging methods promise to yield precise molecular single-cell data and information on cellular neighborhoods and tissue architecture. However, attaining mosaic images with single-cell accuracy requires robust image stitching and image registration capabilities that are not met by existing methods.

Results: We describe the development and testing of ASHLAR, a Python tool for coordinated stitching and registration of 103 or more individual multiplexed images to generate accurate whole-slide mosaics. ASHLAR reads image formats from most commercial microscopes and slide scanners, and we show that it performs better than existing open-source and commercial software. ASHLAR outputs standard OME-TIFF images that are ready for analysis by other open-source tools and recently developed image analysis pipelines.

Availability and implementation: ASHLAR is written in Python and is available under the MIT license at https://github.com/labsyspharm/ashlar. The newly published data underlying this article are available in Sage Synapse at https://dx.doi.org/10.7303/syn25826362; the availability of other previously published data re-analyzed in this article is described in Supplementary Table S4. An informational website with user guides and test data is available at https://labsyspharm.github.io/ashlar/.

Supplementary information: Supplementary data are available at Bioinformatics online.

Fig. 1.

Schematic of cyclic whole-slide data acquisition, stitching and registration. (a) One cycle of whole-slide imaging (scanning) is achieved by moving the microscope stage along a controlled path and acquiring multichannel image tiles that overlap. Further cycles repeat the process after the specimens are re-labeled with new antibodies or other detection reagents. Note that the left-hand portion of this panel depicts just a single reference channel (blue) across three cycles for clarity—actual data contains multiple channels and an arbitrary number of cycles. To integrate information across a wide spatial context at high resolution, it is necessary to stitch neighboring image tiles within one cycle and also register tiles across different cycles. (b) The corners of four neighboring tiles (Hoechst 33342-stained channel, pseudocolored by tile) from one cycle are positioned using the recorded microscope stage positions (upper panel) and the corrected stitched positions (lower panel). Arrows indicate two individual cells in the tile overlap regions before and after stitching. (c) The centers of three Hoechst-channel image tiles (pseudocolored by cycle) from different cycles are positioned using recorded stage positions (upper panel) and post-ASHLAR registered positions (lower panel). Arrows indicate one cell before and after registration (A color version of this figure appears in the online version of this article.)

Fig. 2.

ASHLAR phases for aligning whole-slide scans. (a) Steps for stitching tiles within one cycle. (b) Steps for registering tiles across cycles. (c) Seamless mosaic generation enables whole-slide visualization and flexible re-tiling for downstream parallel processing. Blue-colored graphic components in each step depict the key elements or processes of that step. See text for details (A color version of this figure appears in the online version of this article.)

Fig. 3.

Visualizing stitching steps using a whole-slide scan from a colon specimen. All images and data in this figure derive from analysis of a large multi-tile image of human colon (see text for details). (a) Alignment of one pair of neighboring tiles from an image of human colon. Images of Hoechst 33342-stained nuclei in left and right tiles are pseudocolored in red and cyan, respectively. The red and cyan images are overlaid before and after stitching to demonstrate the effect at the single-cell level. For context, in the remaining panels the location of this tile pair is denoted with a yellow X. (b) Alignment shift distance versus E_NCC for all neighboring pairs, with E_NCC threshold and user-provided translation limit indicated. The null distribution generated by the permutation test (red) is overlaid on the E_NCC marginal distribution. Note that while the E_NCC threshold is computed as the 99th percentile of the null distribution NCC values, it appears at the left end of the null distribution in this figure due to transformation of NCC to E_NCC by taking the negative logarithm (see text). (c) Adjacency graph with edges colored by E_NCC overlaid on the Hoechst image. Edges corresponding to discarded alignments (E_NCC or shift distance above the thresholds) are hidden. Hidden edges correlate with regions containing scant or no tissue. (d) Minimum spanning tree with edges colored by alignment shift distance (A color version of this figure appears in the online version of this article.)

Fig. 4.

ASHLAR mosaic results. All images and data in this figure derive from analysis of the multi-tile image of human colon shown in Figure 3. (a) Pseudocolor image showing five channels from a 28-plex (9-cycle) t-CyCIF image of a normal human colon section acquired using the antibodies described in Supplementary Table S3. Tiles, denoted by the white grid, overlapped by ∼31 pixels (20 µm) Inset: Hematoxylin and eosin (H&E) staining of an adjacent section of the same specimen. (b) Higher magnification view of the area surrounding a single tile showing seven channels from four different cycles to highlight stitching and registration accuracy. Insets 1–4 depict regions of the tile overlap areas at full resolution (note that the antibodies shown in panels a and b differ to make structures relevant to different spatial scales more apparent)

Fig. 5.

Performance comparison of ASHLAR and MIST software. (a) Stitching error metric distributions (kernel density estimate) for MIST and ASHLAR computed according to the stitching evaluation framework of Chalfoun et al. (2017). Dotted lines indicate median values; neither difference in medians was statistically significant. (b–f) Images and data derive from analysis of the multi-tile image of human colon shown in Figures 3 and 4. (b) Local registration error distance distributions for MIST and ASHLAR mosaic images of two t-CyCIF cycles of a human colon section. Distances at the upper end in this plot as well as in panels d and f were clipped to the 90th percentile of the MIST error values (∼4 µm) to highlight the relevant data. (c) Full-resolution view of four regions from the MIST mosaics demonstrating local registration error in different directions. The Hoechst images of nuclei from cycles 1 and 2 are pseudocolored red and cyan, respectively, to visualize the effect of registration error at the single-cell level. The MIST median error of ∼2 µm is around one-quarter of the diameter of the average cell nucleus, a shift that is clearly visible at full resolution. (d) Heatmap of MIST local registration error direction (hue) and magnitude (intensity) at 200-pixel resolution overlaid on the Hoechst image (brighter colors indicate larger errors). Characteristic tile-sized scale of heatmap features suggests inconsistent stitching. Yellow X marks indicate locations highlighted in panel c. (e) The same regions as in panel c, but taken from ASHLAR mosaics. An identical pseudocoloring scheme is used; the red and cyan images, now more accurately registered, combine to appear nearly white. The ASHLAR median registration error of ∼0.1 µm is ∼1% of the diameter of a nucleus. (f) Heatmap of Ashlar local registration error using the same intensity and hue scale as in panel d showing overall lower error and no apparent tile-scale features. Remaining small-scale errors represent damaged tissue that could not be registered (A color version of this figure appears in the online version of this article.)

Fig. 6.

Comparison of registration accuracy between ASHLAR and software included with various commercial slide scanners. All images and data in this figure derive from a single section of a human colorectal adenocarcinoma biopsy (see text for details). (a) Local registration error distance distributions for technical replicate slide scans on three dedicated slide scanning microscopes as well as an ASHLAR mosaic from a research-grade microscope. All scans used the same H&E-stained section of a human colon adenocarcinoma biopsy to allow direct comparison of results. Distances in this panel as well as panels (c–f) were clipped to 1.5 µm at the upper end to highlight the relevant data. (b) H&E staining of the sample used for this analysis. (c) Heatmap of ASHLAR local registration error direction (hue) and magnitude (intensity) at 200-pixel resolution overlaid on the blue channel of the brightfield image, inverted (bright becomes dark and vice versa). (d–f) Heatmap of the three slide scanners’ local registration error, as in panel (c). Engineering details of the different instruments are readily apparent in the error field patterns