WorkflowHub: a registry for computational workflows

Abstract

The rising popularity of computational workflows is driven by the need for repetitive and scalable data processing, sharing of processing know-how, and transparent methods. As both combined records of analysis and descriptions of processing steps, workflows should be reproducible, reusable, adaptable, and available. Workflow sharing presents opportunities to reduce unnecessary reinvention, promote reuse, increase access to best practice analyses for non-experts, and increase productivity. In reality, workflows are scattered and difficult to find, in part due to the diversity of available workflow engines and ecosystems, and because workflow sharing is not yet part of research practice. WorkflowHub provides a unified registry for all computational workflows that links to community repositories, and supports both the workflow lifecycle and making workflows findable, accessible, interoperable, and reusable (FAIR). By interoperating with diverse platforms, services, and external registries, WorkflowHub adds value by supporting workflow sharing, explicitly assigning credit, enhancing FAIRness, and promoting workflows as scholarly artefacts. The registry has a global reach, with hundreds of research organisations involved, and more than 800 workflows registered.

Fig. 1

WorkflowHub connects to platforms, services, and resources that support a workflow’s life cycle. A researcher initially needs to Plan & Find, where they either plan for a particular analysis and find existing workflows (i.e. using a registry), or Develop a new workflow. WorkflowHub integrates with Git repositories (e.g. GitHub, GitLab), and Git-supported communities (e.g. nf-core), to support development. A workflow requires Test & Review to Run & Deploy, and here WorkflowHub connects to support services (e.g. LifeMonitor, bio.tools, Sapporo WES, WfExS) and welcomes diverse workflow platforms that aid deployment (e.g. Common Workflow Language CWL, Snakemake, Galaxy, Jupyter, Python, BASH, WDL, Nextflow). A creator needs to Share a workflow and can benefit from WorkflowHub’s use of citation infrastructures and standards (i.e. CITATION.cff, Zenodo, DataCite, DOI and ORCID). In the Maintain & Learn stage, maintenance, and also understanding of a workflow by other researchers, becomes critical as it impacts workflow Reuse & Rework, where a workflow is either reused, or adapted, by other researchers to suit their requirements. WorkflowHub supports these stages through registration of digital objects that enrich a workflow (e.g. documents, publications, SOPs), the ability to create Collections and workflow citations based on DOIs, and ultimately through the connections created to knowledge graphs. WorkflowHub also enables communities of practice to benefit from all its integrations and connections, ensuring that they can reuse or rework workflows from across the globe. The entire support framework is enabled by the implementation of standards that allow WorkflowHub to interact with the ecosystem and truly act as a “Hub”: EDAM, Research Object Crates (RO-Crates), Global Alliance for Genomics and Health (GA4GH) application programming interfaces (APIs), abstract CWL, FAIR Signposting, and Bioschemas.

Fig. 2

Breakdown of discipline origins for the groups (also known as Teams) that have registered with WorkflowHub. Teams were categorised by the registry maintainers.

Fig. 3

Workflow types registered with WorkflowHub. See https://workflowhub.eu/workflow_classes for more details.

Fig. 4

A guide to the structures in WorkflowHub. You, the user, belong to one or more Organisations (i.e. affiliations). You can also belong to one or more Teams, each of which also needs to belong to a single Space (top). You can nominate which Organisations you wish to use for the different Teams that you have created or joined, and you can belong to multiple Teams in the same Space, as well as multiple Teams in other Spaces (bottom). Image reused with permission from WorkflowHub documentation.

Fig. 5

WorkflowHub entry for the dna-seq-varlociraptor Snakemake workflow with annotations for different sections of the user interface. Features include (A) workflow type, (B) title, (C) access panel with links to the source repository (e.g. GitHub), requests to contact the creators, subscribe / unsubscribe, download Research Object Crate (RO-Crate), and add to a Collection, (D) tabs for navigation between the entry overview, the list of files in the entry, and lists of items related to the workflow, including people, Teams, Spaces, Organisations, and other digital objects (e.g. publications, documents, SOPs, other workflows), (E) description, which can be imported from Git (if available), (F) version history, including Git commits, (if available), (G) creator and submitter information, (H) links to more information about tools that comprise the workflow (i.e. bio.tools registry entries), (I) license information, (J) activity metrics (i.e. downloads and views), and (K) ontology concept annotations (i.e. EDAM).

Fig. 6

WorkflowHub entry for the Find transcripts - TSI Galaxy workflow with additional entry specific annotations. Features unique to this entry include (A) a workflow diagram, (B) parsed workflow inputs, outputs and steps for the registered Galaxy workflow, (C) a citation for the workflow, (D) custom tags, (E) Collections that include the current workflow entry, and (F) a button for launching the workflow on Galaxy.