Systematic comparison of sequencing-based spatial transcriptomic methods

Abstract

Recent developments of sequencing-based spatial transcriptomics (sST) have catalyzed important advancements by facilitating transcriptome-scale spatial gene expression measurement. Despite this progress, efforts to comprehensively benchmark different platforms are currently lacking. The extant variability across technologies and datasets poses challenges in formulating standardized evaluation metrics. In this study, we established a collection of reference tissues and regions characterized by well-defined histological architectures, and used them to generate data to compare 11 sST methods. We highlighted molecular diffusion as a variable parameter across different methods and tissues, significantly affecting the effective resolutions. Furthermore, we observed that spatial transcriptomic data demonstrate unique attributes beyond merely adding a spatial axis to single-cell data, including an enhanced ability to capture patterned rare cell states along with specific markers, albeit being influenced by multiple factors including sequencing depth and resolution. Our study assists biologists in sST platform selection, and helps foster a consensus on evaluation standards and establish a framework for future benchmarking efforts that can be used as a gold standard for the development and benchmarking of computational tools for spatial transcriptomic analysis.

Fig. 1. Overview of experimental design and data processing pipeline.

a, The experimental design involved the use of reference tissues, with technologies categorized by the distinct spatial indexing strategies. b, The visualization of total counts across the spatial dimension for datasets generated using each platform for reference tissues is shown. The distances from center to center, used in creating the plot, are presented alongside the name of each sST method. Scale bars, 500 μm.

Fig. 2. Comparison of the sensitivity of data generated by different platforms.

a, Schematic plot illustrating the extraction of regions with known morphology from fully processed samples of the adult mouse hippocampus and E12.5 mouse eye. Total UMI counts are presented as a function of stepwise downsampled sequencing depths for each platform. b,c, The data originate from mouse hippocampus (b) and E12.5 mouse eye (c) regions. A vertical dashed black line marks the read count used for generating the subsequent downsampled data. d, Total UMI counts were computed for selected regions using all reads and downsampled data for the mouse hippocampus. e, Total UMI counts for selected regions using all reads and downsampled data for the E12.5 mouse eye. f, The summed UMI counts for marker genes across individual 50 × 50 μm regions (n = 4) in the mouse hippocampus, along with mean and standard deviation. g, The summed UMI counts for marker genes across individual 50 × 50 μm regions (n = 4) in the E12.5 mouse eye, along with mean and standard deviation. h, Total UMI counts of detected genes are compared between Visium(polyA) (x axis) and Stereo-seq (y axis). Each dot represents a gene, shown in black. Genes that display expression at the 90th percentile with Stereo-seq but are at the tenth percentile in Visium(polyA) are highlighted in red and labeled with their gene symbols. i, A heatmap displays the ${\log }_{10}$-transformed expression of genes that are specifically not captured by Visium(polyA) but are captured by Stereo-seq for E12.5 mouse eyes.

Fig. 3. Comparison of diffusion of data generated by different platforms.

a, Expression markers include Slc17a7 in the mouse olfactory bulb (left), Ptgds in the mouse brain (middle) and Pmel in the E12.5 eye (right). The plots are based on raw count values. Black boxes indicate the selected regions used for diffusion calculation. b, Expression levels of the aforementioned marker genes (from a) are aggregated for every 10 μm along 50 μm in the olfactory bulb, 500 μm in the brain and 300 μm in the eyes, as shown in a. UMI counts are averaged across modalities, normalized for each platform and presented in a density plot with the area under the curve set to 1 (details in Methods). c, Expression level of the marker genes as mentioned above (from a) within selected modalities are provided, with black dashed lines delineating the boundaries used for diffusion calculations. d, The left-width at half-maximum (LWHM) of the profile was then calculated for each gene (from a) in each modality and displayed in boxplots. Each dot represents the LWHM for a given modality (n = 6 for olfactory bulb and brain, n = 3 for eye). Modalities for which LWHM could not be calculated were removed. The box denotes the interquartile range, the range between the 25th and 75th percentile, with the median value; whiskers indicate the maximum and minimum value within 1.5 times the interquartile range.

Fig. 4. Comparison on downstream performance.

a, Expression profiles generated by each platform were processed to obtain clustering results. Known cell types and states are colored in the left-most panel. Additionally, a schematic plot represents the expected cell states, arranged from the outer space to inner space and from top to bottom. On the right-hand side, clustering results are presented, with spots color-coded by annotated cell states depicting the identifiable cell states. b, Clustering was conducted on downsampled eye data from each platform, with an equal total read count across platforms in the eye area. The correspondence between annotations obtained from clustering based on all reads and clustering based on downsampled data is visualized in a heatmap. The number of spots in this correspondence is presented after ${\log }_{10}$ transformation without scaling. c, An overview of cell states compared in the marker gene detection analysis, with pNR4 and pNR1 highlighted. d, Number of marker genes detected with different numbers of reads used for each sST method in the comparison between pNR4 and pNR1. e, An upset plot displays the intersection of marker genes obtained by different sST methods using all reads for the pNR4 and pNR1 comparison. Genes shared among all three platforms are denoted in blue, those shared between two platforms are in purple and uniquely obtained genes are represented in pink. f, The sST methods have been ranked based on their performance in the specified categories, with the highest-performing methods positioned at the top. Methods that offer resolution levels below 20 μm have been given higher preference. In the right panel, essential characteristics of the sST methods examined are outlined. Lower affordability indicates a higher price associated with the method. CM, corneal mesenchyme; pNR, presumptive neural retina; LV, lens vesicle; OB, olfactory bulb.