Systematic comparison of single-cell and single-nucleus RNA-sequencing methods

Abstract

The scale and capabilities of single-cell RNA-sequencing methods have expanded rapidly in recent years, enabling major discoveries and large-scale cell mapping efforts. However, these methods have not been systematically and comprehensively benchmarked. Here, we directly compare seven methods for single-cell and/or single-nucleus profiling-selecting representative methods based on their usage and our expertise and resources to prepare libraries-including two low-throughput and five high-throughput methods. We tested the methods on three types of samples: cell lines, peripheral blood mononuclear cells and brain tissue, generating 36 libraries in six separate experiments in a single center. To directly compare the methods and avoid processing differences introduced by the existing pipelines, we developed scumi, a flexible computational pipeline that can be used with any single-cell RNA-sequencing method. We evaluated the methods for both basic performance, such as the structure and alignment of reads, sensitivity and extent of multiplets, and for their ability to recover known biological information in the samples.

Extended Data Figure 1

Extended Data Figure 2

Extended Data Figure 3

Extended Data Figure 4

Extended Data Figure 5

Extended Data Figure 6

Extended Data Figure 7

Extended Data Figure 8

Extended Data Figure 9

Extended Data Figure 10

Figure 1.. Study Overview.

(a) samples, (b) scRNA-seq methods, (c) computational pipeline summary. Cell line mixtures tested with all methods. PBMCs tested with all methods except sci-RNA-seq. Cortex nuclei tested with Smart-seq2, 10x Chromium, Drop-seq (aka DroNc-seq for nuclei), and sci-RNA-seq. Additional details can be found in Extended Data Figs. 1 and 2.

Figure 2.. Performance metrics for mixture experiments.

(a-b) Distribution of the number of UMIs (a) or genes (b) in human (top) or mouse (bottom) cells in the two Mixture experiments (n=1 biologically independent sample per experiment). For (a) and (b), median and box plots were based on all the cells, but a few outlier cells were omitted in drawing the violin plots. Box plots denote the medians (labeled on the right) and the interquartile ranges (IQRs). The whiskers of each boxplot are the lowest datum still within 1.5 IQR of the lower quartile and the highest datum still within 1.5 IQR of the upper quartile. Violin plot width is based on a Gaussian kernel density estimate of the data (estimated by the density function with standard parameters), scaled to have maximum width = 1. (c) Multiplet frequency. We ordered cells based on the number of detected UMIs (or reads for Smart-seq2), from highest (left) to lowest (right). For a given number of cells (x-axis value), the plot shows the percent of cells that are multiplets. The dotted lines for sci-RNA-seq Mixture1 and inDrops Mixture1 and Mixture2 show the multiplet rate including low-quality cells that were not included in subsequent analysis.

Figure 3.. PBMCs sensitivity.

Distribution of the number of UMIs (a) or genes (b) per cell for each method in the two experiments (n=1 biologically independent sample per experiment). Violin and box plot elements are defined as in Fig. 2.

Figure 4.. Cortex nuclei sensitivity.

Distribution of the number of UMIs (a) or genes (b) per cell for each method in the two experiments (n=1 biologically independent sample per experiment). Violin and box plot elements are defined as in Fig. 2.

Figure 5.. Cell type identification and assignment in PBMCs.

(a) t-stochastic neighborhood embeddings (t-SNEs) of single cell profiles (dots) from representative PBMC2 libraries colored by cell type. (b) Proportion of cells of each cell type (y-axis) detected with different methods (x-axis). Those not labeled with a number rounded to one or less. Sum does not always add to 100 due to this and rounding. (c) The AUC (dot size, color and value) of each cluster from classifying the cell type to the cluster it was assigned for PBMC1 and PBMC2. See Supplementary Table 2 for the numbers of cells used (n=1 biologically independent sample per experiment).

Figure 6.. Cell type identification and assignment in cortex nuclei.

(a) t-SNEs of single cell profiles (dots) from Cortex1 libraries colored by cell type. (b) Proportion of cells of each cell type (y-axis) detected with different methods (x-axis). (c) The AUC (dot size, color and value) of each cluster from classifying the cell type to the cluster it was assigned for Cortex1 and Cortex2. See Supplementary Table 2 for the numbers of cells used (n=1 biologically independent sample per experiment). We could not confidently assign cell types to some clusters of cells from sci-RNA-seq and these cells were not used in calculating the AUCs.