A comprehensive assessment of N-terminal signal peptides prediction methods

Abstract

Background: Amino-terminal signal peptides (SPs) are short regions that guide the targeting of secretory proteins to the correct subcellular compartments in the cell. They are cleaved off upon the passenger protein reaching its destination. The explosive growth in sequencing technologies has led to the deposition of vast numbers of protein sequences necessitating rapid functional annotation techniques, with subcellular localization being a key feature. Of the myriad software prediction tools developed to automate the task of assigning the SP cleavage site of these new sequences, we review here, the performance and reliability of commonly used SP prediction tools.

Results: The available signal peptide data has been manually curated and organized into three datasets representing eukaryotes, Gram-positive and Gram-negative bacteria. These datasets are used to evaluate thirteen prediction tools that are publicly available. SignalP (both the HMM and ANN versions) maintains consistency and achieves the best overall accuracy in all three benchmarking experiments, ranging from 0.872 to 0.914 although other prediction tools are narrowing the performance gap.

Conclusion: The majority of the tools evaluated in this study encounter no difficulty in discriminating between secretory and non-secretory proteins. The challenge clearly remains with pinpointing the correct SP cleavage site. The composite scoring schemes employed by SignalP may help to explain its accuracy. Prediction task is divided into a number of separate steps, thus allowing each score to tackle a particular aspect of the prediction.

Figure 1

Aggregated results from all three experiments. Accuracy results from all three experiments are provided here. For each tool, there are three bars, representing each experiment (gray bar: experiment 1; white bar: experiment 2; black bar: experiment 3).

Figure 2

Results from Experiment 1. The dataset [20] used in this experiment contains eukaryotic (human) sequences only. The bars colored in light gray represent the specificity while the black bars represent the sensitivity of the prediction tools.

Figure 3

Results from Experiment 2. The datasets employed in this experiment are derived from SPdb 5.1 [33] and subjected to manual curation. The datasets are divided into Euk (top chart), Gpos (bottom chart) and Gneg (middle chart) bacteria.

Figure 4

Results from Experiment 3. The datasets employed in this experiment are derived from Swiss-Prot Release 57.0 and subjected to the filtering process described in [33]. However, putative SPs which have high probability of existent based on the experiment literature are retained. The datasets are further grouped into Euk (top chart), Gpos (bottom chart) and Gneg (middle chart) bacteria.