The development of PIPA: an integrated and automated pipeline for genome-wide protein function annotation

Abstract

Background: Automated protein function prediction methods are needed to keep pace with high-throughput sequencing. With the existence of many programs and databases for inferring different protein functions, a pipeline that properly integrates these resources will benefit from the advantages of each method. However, integrated systems usually do not provide mechanisms to generate customized databases to predict particular protein functions. Here, we describe a tool termed PIPA (Pipeline for Protein Annotation) that has these capabilities.

Results: PIPA annotates protein functions by combining the results of multiple programs and databases, such as InterPro and the Conserved Domains Database, into common Gene Ontology (GO) terms. The major algorithms implemented in PIPA are: (1) a profile database generation algorithm, which generates customized profile databases to predict particular protein functions, (2) an automated ontology mapping generation algorithm, which maps various classification schemes into GO, and (3) a consensus algorithm to reconcile annotations from the integrated programs and databases.PIPA's profile generation algorithm is employed to construct the enzyme profile database CatFam, which predicts catalytic functions described by Enzyme Commission (EC) numbers. Validation tests show that CatFam yields average recall and precision larger than 95.0%. CatFam is integrated with PIPA. We use an association rule mining algorithm to automatically generate mappings between terms of two ontologies from annotated sample proteins. Incorporating the ontologies' hierarchical topology into the algorithm increases the number of generated mappings. In particular, it generates 40.0% additional mappings from the Clusters of Orthologous Groups (COG) to EC numbers and a six-fold increase in mappings from COG to GO terms. The mappings to EC numbers show a very high precision (99.8%) and recall (96.6%), while the mappings to GO terms show moderate precision (80.0%) and low recall (33.0%). Our consensus algorithm for GO annotation is based on the computation and propagation of likelihood scores associated with GO terms. The test results suggest that, for a given recall, the application of the consensus algorithm yields higher precision than when consensus is not used.

Conclusion: The algorithms implemented in PIPA provide automated genome-wide protein function annotation based on reconciled predictions from multiple resources.

Figure 1

Overview of PIPA's key modules. PIPA's programs are organized into three modules. The pipeline execution module consists of programs that enable user access to and control of the pipeline's parallel execution of multiple programs. The execution module wraps the core module, containing all integrated methods (programs and databases), the terminology conversion program, and the consensus annotation program. The support module contains the profile database generation program, which creates new profile databases, and the GO-mapping generation program, which creates GO mappings for the terminology conversion program.

Figure 2

CatFam performance evaluation. The performance of CatFam is measured by precision and recall, which are defined as precision = TP/(TP+FP) and recall = TP/(TP+FN), where TP, FP and FN represent the number of true-positive, false-positive, and false-negative predictions, respectively. A total of 18,949 proteins, not used for profile generation, are used to evaluate two CatFam databases, CatFam-3D and CatFam-4D, which predict 3-digit and 4-digit EC numbers for query proteins, respectively. The results are sorted according to the maximum sequence identity between the query protein and the proteins used for profile generation.

Figure 3

Mapping evaluation. The number of automatically generated mappings is significantly increased for properly-selected cut-off E-values when the hierarchical topology of the ontologies is used. Larger cut-off E-values (small values on the x axis) result in excessive false hits for sample proteins, while smaller cut-off E-values exclude true hits. Both cases reduce the number of mappings that can be generated.

Figure 4

GO consensus evaluation. PIPA's GO annotations with and without the consensus algorithm are compared for hierarchical precision (HP) and hierarchical recall (HR). For the consensus algorithm, each data point in the figure is computed by using different combinations of the three parameters, E₀, E₁, and SAT. When the consensus algorithm is not used, each data point is obtained by selecting a different cut-off E-value. The figure indicates that the consensus algorithm improves HP for each of the three GO categories. The highlighted points correspond to consensus algorithm results with parameters E₀ = 0.01, E₁ = 1e-200, and SAT = 0.99.