Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses

Abstract

Due to the upcoming data deluge of genome data, the need for storing and processing large-scale genome data, easy access to biomedical analyses tools, efficient data sharing and retrieval has presented significant challenges. The variability in data volume results in variable computing and storage requirements, therefore biomedical researchers are pursuing more reliable, dynamic and convenient methods for conducting sequencing analyses. This paper proposes a Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses, which enables reliable and highly scalable execution of sequencing analyses workflows in a fully automated manner. Our platform extends the existing Galaxy workflow system by adding data management capabilities for transferring large quantities of data efficiently and reliably (via Globus Transfer), domain-specific analyses tools preconfigured for immediate use by researchers (via user-specific tools integration), automatic deployment on Cloud for on-demand resource allocation and pay-as-you-go pricing (via Globus Provision), a Cloud provisioning tool for auto-scaling (via HTCondor scheduler), and the support for validating the correctness of workflows (via semantic verification tools). Two bioinformatics workflow use cases as well as performance evaluation are presented to validate the feasibility of the proposed approach.

Keywords: Bioinformatics; Cloud computing; Galaxy; Scientific workflow; Sequencing analyses.

Fig. 1

RNA-Sequencing analysis workflow.

Fig. 2

Globus Transfer tools in Galaxy.

Fig. 3

CRData tools in Galaxy (this figure shows the interface of “sequenceDifferentialExperssion.R”).

Fig. 4

CummeRbund tool in Galaxy.

Fig. 5

Density plot generated by CummeRbund.

Fig. 6

Connection of Cuffdiff and CummeRbund.

Fig. 7

The main steps for using Globus Provision (the blocks with solid lines are necessary steps, while the ones with dashed lines are optional steps).

Fig. 8

Topology file “galaxy.conf”.

Fig. 9

Architecture of Cloud-based bioinformatics workflow platform.

Fig. 10

Security mechanism.

Fig. 11

CRData workflow.

Fig. 12

CRData tool “affyDifferentialExpression.R” (Step 3).

Fig. 13

Text output of “affyDifferentialExpression.R” (Step 3).

Fig. 14

Figure output of “affyDifferentialExpression.R” in Step 3 (a) and Step 4 (b).

Fig. 15

The RNA-Seq analysis workflow in Galaxy.

Fig. 16

The execution time and cost of RNA-Seq workflow with different EC2 instance types.