A systematic evaluation of the computational tools for ligand-receptor-based cell-cell interaction inference

Abstract

Cell-cell interactions (CCIs) are essential for multicellular organisms to coordinate biological processes and functions. One classical type of CCI interaction is between secreted ligands and cell surface receptors, i.e. ligand-receptor (LR) interactions. With the recent development of single-cell technologies, a large amount of single-cell ribonucleic acid (RNA) sequencing (scRNA-Seq) data has become widely available. This data availability motivated the single-cell-resolution study of CCIs, particularly LR-based CCIs. Dozens of computational methods and tools have been developed to predict CCIs by identifying LR-based CCIs. Many of these tools have been theoretically reviewed. However, there is little study on current LR-based CCI prediction tools regarding their performance and running results on public scRNA-Seq datasets. In this work, to fill this gap, we tested and compared nine of the most recent computational tools for LR-based CCI prediction. We used 15 well-studied scRNA-Seq samples that correspond to approximately 100K single cells under different experimental conditions for testing and comparison. Besides briefing the methodology used in these nine tools, we summarized the similarities and differences of these tools in terms of both LR prediction and CCI inference between cell types. We provided insight into using these tools to make meaningful discoveries in understanding cell communications.

Keywords: cell–cell interaction; computational prediction tools; ligand-receptor interaction; single-cell RNA sequencing.

Figure 1

The overlap of the LR pairs in the 23 resources. Although many LR interactions are shared among these resources, much more LR interactions are unique to individual resources.

Figure 2

Tool results on different datasets. (A) The number of predicted LR pairs varies from FIB-A to FIB-B cell types in the embryonic mouse skin samples. (B) A large percentage of predicted LR pairs by all tools except PyMINEr are in C-LRI from FIB-A to FIB-B cell types in embryonic mouse skin samples. (C) The number of predicted LR pairs varies from pyramidal CA1 to oligodendrocytes cell types in mouse cerebral cortex. (D) A large percentage of predicted LR pairs by all tools except PyMINEr are in C-LRI from pyramidal CA1 to oligodendrocytes cell types in mouse cerebral cortex.

Figure 3

The number of LR pairs between pyramidal CA1 and pyramidal SS cell types in mouse cerebral cortex consistently increases with the higher resolution setting, except SingleCellSignalR.

Figure 4

The CCI network dissimilarity scores between different tools on the four samples in the embryonic mouse skin dataset. A subset of tools have similar CCI outputs (darker blocks), while scMLnet and SingleCellSignalR have quite different CCI output from other tools.

Figure 5

CCI network examples of the selected six tools on the embryonic mouse skin dataset (GSM3453536). SingleCellSingalR and scMLnet generally have much smaller edge weights and fewer CCI edges between cells than other tools. (A) The CCI subnetwork corresponding to the basal, immune, FIB-A and FIB-B cell types. (B) The CCI subnetwork corresponding to the endothelial, spinous, MELA and muscle cell types.

Figure 6

ICELLNET and NATMI have higher subsampling consistency scores (precision, recall and F1 score) than other tools. (A) The subsampling consistency scores on the Embryonic mouse skin data. (B) The subsampling consistency scores on the Mouse cerebral cortex data.

Figure 7

Comparative performance of tools on the mouse spine cord injury dataset. (A) The number of predicted LR pairs from microglia to endothelial cell types varies greatly. (B) A large proportion of predicted LR pairs by all tools except PyMINEr are in C-LRI from microglia to endothelial cell types.

Figure 8

The comparison of the running time and memory usage of the nine tools.