Systematic benchmarking of imaging spatial transcriptomics platforms in FFPE tissues

Abstract

Emerging imaging spatial transcriptomics (iST) platforms and coupled analytical methods can recover cell-to-cell interactions, groups of spatially covarying genes, and gene signatures associated with pathological features, and are thus particularly well-suited for applications in formalin fixed paraffin embedded (FFPE) tissues. Here, we benchmark the performance of three commercial iST platforms-10X Xenium, Vizgen MERSCOPE, and Nanostring CosMx-on serial sections from tissue microarrays (TMAs) containing 17 tumor and 16 normal tissue types for both relative technical and biological performance. On matched genes, we find that Xenium consistently generates higher transcript counts per gene without sacrificing specificity. Xenium and CosMx measure RNA transcripts in concordance with orthogonal single-cell transcriptomics. All three platforms can perform spatially resolved cell typing with varying degrees of sub-clustering capabilities, with Xenium and CosMx finding slightly more clusters than MERSCOPE, albeit with different false discovery rates and cell segmentation error frequencies. Taken together, our analyses provide a comprehensive benchmark to guide the choice of iST method as researchers design studies with precious samples in this rapidly evolving field.

Fig. 1. Experimental design and iST platforms.

a Overall approach for generating iST data. b Different amplification approaches for Xenium, MERSCOPE, and CosMx. c Overview of the tissue types and numbers of cores used in this study. BlC bladder cancer, BrC breast cancer, CRC colorectal cancer, HNSCC head and neck squamous cell carcinoma, Mel Melanoma, NSCLC non-small cell lung cancer, OvC ovarian cancer, SCC squamous cell carcinoma. d DAPI images from the Xenium run of each TMA, including Tumor TMA 1(tTMA1), Tumor TMA 2(tTMA2), and Normal TMA(nTMA).

Fig. 2. Technical performance comparison of iST platforms using 2024 datasets.

a Scatter plots of summed gene expression levels (on a logarithmic scale) of every shared gene between Xenium (breast) and CosMx (1k) data, captured from matched cores from tTMA1(24). Each data point corresponds to a gene. The red line represents the fitted regression (center), obtained from a first-order polynomial fit in log–log space and back-transformed to the original scale. The shaded band shows ±1.96× the standard deviation of residuals (linear scale) around this line. b Same as (a) but between MERSCOPE (breast) and CosMx(1k). c Same as (a) but between Xenium(breast) and MERSCOPE(breast). d Violin plot of percentage of all transcripts corresponding to genes relative to the total number of calls (including negative control probes and unused barcodes) averaged across cores of the same tissue type. Results are from Xenium’s breast panel; MERSCOPE’s breast panel; and CosMx’s multi-tissue 1k panel. Each data point represents a TMA-tissue type combination, such as tTMA1(24)-BrC or tTMA2(24)-BlC. Violins show kernel density; interior lines denote quartiles (median = 2nd quartile). The full data is shown in Supplementary Fig. 5a. e Violin plot of false discovery rate (FDR) where FDR(%) = (blank barcode calls / total transcript calls) x (Number of panel genes /Number of blank barcode) x 100. f Same as (e) but using negative control probes to replace blank barcodes. MERSCOPE is missing in this bar plot as it does not have negative control probes by design. g Violin plot of number of genes detected above noise, estimated as two standard deviations above average expression of the negative control probes. h Same as (g) but normalized to the number of genes in a panel or in percentage. Pairwise differences between platforms were assessed with two-sided Mann–Whitney U tests for (d–h); brackets show unadjusted p-values for each comparison.

Fig. 3. Concordance of iST data with reference RNA-seq datasets.

a Scatter plots of common genes, showing the averaged expression of a gene across breast cancer cores profiled by the indicated panel, normalized to 100,000 vs the average FPKM from TCGA for all samples of a matched tissue type from tTMA1 (24). 1st-order polynomial fitting was performed and is shown as a black line. Insets are Spearman correlation coefficients. b Same as (a) but showing breast cores from nTMA (23) dataset versus averaged nTPM values from GTEx breast samples. c Scatter plots of overlapping genes, showing the aggregated expression of a gene in tTMA1 (23) across smooth muscle cells profiled by the indicated panel vs the gene expression from scRNA-seq. d Same as (c) but for tTMA1 (24). e Same as (c) but for tTMA2 (24). f Comparisons of residuals between single-cell and tTMA1 (23)/tTMA1 (24) for all platforms.

Fig. 4. Comparison of cell segmentation results from each iST platform.

a Top row: Subset of data showing DAPI (blue fill) and membrane staining (green fill) overlaid with cell segmentation boundaries (white outline) and manually annotated cell centroid (red point). Middle row: all the transcripts in green dots and EPCAM in blue dots. Bottom row: segmented cell boundaries (white outline) before and after filtration (Cyan outline: Cells kept after quality control; Orange outline: Cells excluded after quality control). We acquired imaging data from 263 TMA cores (170 from tTMA1, 48 from tTMA2, and 45 from nTMA) using Xenium, MERSCOPE, and CosMx, respectively. Segmentation was performed on three representative TMA cores for each IST platform, yielding a total of 9 cores and 31,384 annotated cells. b Segmentation accuracy evaluated by three metrics: Precision, Recall, and F1 Score, in various scenarios including dense cells, sparse cells, and elongated cells. Pairwise platform differences were tested using two-sided Tukey’s HSD following one-way grouping by platform (per core and scenario). Reported p-values are Tukey-adjusted for multiple comparisons. Asterisks indicate significance thresholds (*p < 0.05; **p < 0.01; ***p < 0.001; ****p < 0.0001). Exact adjusted p-values are shown above brackets. c Heatmap of transcripts per cell after filtration. All available genes are considered here for each panel. We filtered out cells with fewer than 10 transcripts for Xenium and MERSCOPE, and fewer than 20 transcripts for CosMx, in accordance with each platform’s recommended threshold. d Same as (c) but showing unique genes per cell. e Same as (c) but reanalyzed using only shared genes. f Same as (d) but reanalyzed using only shared genes. g Co-expression density map for three pairs of disjoint genes (rows) from all three platforms (columns) from tTMA1 (24). MERSCOPE breast dataset does not have enough cells to generate the 2D histogram for PPARG vs. CD68. All cells across all tissues which include at least one detected transcript of either of the indicated genes are plotted together, with color indicating the number of cells at the indicated expression levels of each gene. Data throughout is from tTMA1 (24) and tTMA2 (24).

Fig. 5. Cell type recovery performance across technology.

a Clustering results of breast cancer samples in tTMA1 (23) from Xenium breast panel, MERSCOPE breast panel, and CosMx multi-tissue panel. Correlation plot showing the correlation between cell types identified in CosMx and Xenium as well as MERSCOPE and Xenium. b Clustering results of breast cancer samples in tTMA1 (24) from Xenium breast panel, MERSCOPE breast panel, and CosMx multi-tissue panel. Correlation plot showing the correlation between cell types identified in CosMx and Xenium as well as MERSCOPE and Xenium. c Heatmaps show high cell type annotation correlation between experiments conducted for Tumor TMA 1 of Xenium (top), MERSCOPE (middle), CosMx (bottom). d Clustering results of breast cancer samples in tTMA2 (24) from Xenium breast panel and CosMx multi-tissue panel. Correlation plot showing the correlation between cell types identified in CosMx and Xenium.