Developing reproducible bioinformatics analysis workflows for heterogeneous computing environments to support African genomics

doi:10.1186/s12859-018-2446-1

. 2018 Nov 29;19(1):457.

doi: 10.1186/s12859-018-2446-1.

Developing reproducible bioinformatics analysis workflows for heterogeneous computing environments to support African genomics

Shakuntala Baichoo¹, Yassine Souilmi², Sumir Panji³, Gerrit Botha³, Ayton Meintjes³, Scott Hazelhurst^{4

5}, Hocine Bendou^{6

7}, Eugene de Beste^{6

7}, Phelelani T Mpangase⁵, Oussema Souiai^{8

9}, Mustafa Alghali^{10

11}, Long Yi^{6

7}, Brian D O'Connor¹², Michael Crusoe¹³, Don Armstrong¹⁴, Shaun Aron⁵, Fourie Joubert¹⁵, Azza E Ahmed^{10

11}, Mamana Mbiyavanga³, Peter van Heusden^{6

7}, Lerato E Magosi^{16

17}, Jennie Zermeno¹⁸, Liudmila Sergeevna Mainzer^{14

18}, Faisal M Fadlelmola¹⁰, C Victor Jongeneel¹⁴, Nicola Mulder³

Affiliations

¹ Department of Digital Technologies, University of Mauritius, Reduit, Mauritius. shakunb@uom.ac.mu.
² Australian Centre for Ancient DNA, University of Adelaide, Adelaide, South Australia, Australia.
³ Computational Biology Division, Department of Integrative Medical Biosciences, IDM, University of Cape Town, Cape Town, South Africa.
⁴ School of Electrical & Information Engineering, University of the Witwatersrand, Johannesburg, South Africa.
⁵ Sydney Brenner Institute for Molecular Bioscience, University of the Witwatersrand, Johannesburg, South Africa.
⁶ South African National Bioinformatics Institute, University of the Western Cape, Bellville, Cape Town, South Africa.
⁷ Natural Sciences, University of the Western Cape, Bellville, Cape Town, South Africa.
⁸ Institut Pasteur De Tunis, University Tunis El manar, Tunis, Tunisia.
⁹ Institut Superieur des Technologies Medicales de Tunis, University Tunis El manar, Tunis, Tunisia.
¹⁰ Center for Bioinformatics & Systems Biology, Faculty of Science, University of Khartoum, Khartoum, Sudan.
¹¹ Department of Electrical & Electronic Engineering, Faculty of Engineering, University of Khartoum, Khartoum, Sudan.
¹² Genomics Institute, University of California, Santa Cruz, California, USA.
¹³ Common Workflow Language project, Software Freedom Conservancy, New York City, NY, USA.
¹⁴ Institute for Genomic Biology, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA.
¹⁵ Centre for Bioinformatics and Computational Biology, Department of Biochemistry, Genetics and Microbiology, University of Pretoria, Pretoria, South Africa.
¹⁶ Wellcome Trust Centre for Human Genetics, University of Oxford, Oxford, UK.
¹⁷ Radcliffe Department of Medicine, University of Oxford, Oxford, UK.
¹⁸ National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA.

PMID: 30486782
PMCID: PMC6264621
DOI: 10.1186/s12859-018-2446-1

Developing reproducible bioinformatics analysis workflows for heterogeneous computing environments to support African genomics

Shakuntala Baichoo et al. BMC Bioinformatics. 2018.

. 2018 Nov 29;19(1):457.

doi: 10.1186/s12859-018-2446-1.

Authors

Affiliations

¹ Department of Digital Technologies, University of Mauritius, Reduit, Mauritius. shakunb@uom.ac.mu.
² Australian Centre for Ancient DNA, University of Adelaide, Adelaide, South Australia, Australia.
³ Computational Biology Division, Department of Integrative Medical Biosciences, IDM, University of Cape Town, Cape Town, South Africa.
⁴ School of Electrical & Information Engineering, University of the Witwatersrand, Johannesburg, South Africa.
⁵ Sydney Brenner Institute for Molecular Bioscience, University of the Witwatersrand, Johannesburg, South Africa.
⁶ South African National Bioinformatics Institute, University of the Western Cape, Bellville, Cape Town, South Africa.
⁷ Natural Sciences, University of the Western Cape, Bellville, Cape Town, South Africa.
⁸ Institut Pasteur De Tunis, University Tunis El manar, Tunis, Tunisia.
⁹ Institut Superieur des Technologies Medicales de Tunis, University Tunis El manar, Tunis, Tunisia.
¹⁰ Center for Bioinformatics & Systems Biology, Faculty of Science, University of Khartoum, Khartoum, Sudan.
¹¹ Department of Electrical & Electronic Engineering, Faculty of Engineering, University of Khartoum, Khartoum, Sudan.
¹² Genomics Institute, University of California, Santa Cruz, California, USA.
¹³ Common Workflow Language project, Software Freedom Conservancy, New York City, NY, USA.
¹⁴ Institute for Genomic Biology, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA.
¹⁵ Centre for Bioinformatics and Computational Biology, Department of Biochemistry, Genetics and Microbiology, University of Pretoria, Pretoria, South Africa.
¹⁶ Wellcome Trust Centre for Human Genetics, University of Oxford, Oxford, UK.
¹⁷ Radcliffe Department of Medicine, University of Oxford, Oxford, UK.
¹⁸ National Center for Supercomputing Applications, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA.

PMID: 30486782
PMCID: PMC6264621
DOI: 10.1186/s12859-018-2446-1

Abstract

Background: The Pan-African bioinformatics network, H3ABioNet, comprises 27 research institutions in 17 African countries. H3ABioNet is part of the Human Health and Heredity in Africa program (H3Africa), an African-led research consortium funded by the US National Institutes of Health and the UK Wellcome Trust, aimed at using genomics to study and improve the health of Africans. A key role of H3ABioNet is to support H3Africa projects by building bioinformatics infrastructure such as portable and reproducible bioinformatics workflows for use on heterogeneous African computing environments. Processing and analysis of genomic data is an example of a big data application requiring complex interdependent data analysis workflows. Such bioinformatics workflows take the primary and secondary input data through several computationally-intensive processing steps using different software packages, where some of the outputs form inputs for other steps. Implementing scalable, reproducible, portable and easy-to-use workflows is particularly challenging.

Results: H3ABioNet has built four workflows to support (1) the calling of variants from high-throughput sequencing data; (2) the analysis of microbial populations from 16S rDNA sequence data; (3) genotyping and genome-wide association studies; and (4) single nucleotide polymorphism imputation. A week-long hackathon was organized in August 2016 with participants from six African bioinformatics groups, and US and European collaborators. Two of the workflows are built using the Common Workflow Language framework (CWL) and two using Nextflow. All the workflows are containerized for improved portability and reproducibility using Docker, and are publicly available for use by members of the H3Africa consortium and the international research community.

Conclusion: The H3ABioNet workflows have been implemented in view of offering ease of use for the end user and high levels of reproducibility and portability, all while following modern state of the art bioinformatics data processing protocols. The H3ABioNet workflows will service the H3Africa consortium projects and are currently in use. All four workflows are also publicly available for research scientists worldwide to use and adapt for their respective needs. The H3ABioNet workflows will help develop bioinformatics capacity and assist genomics research within Africa and serve to increase the scientific output of H3Africa and its Pan-African Bioinformatics Network.

Keywords: Africa; Bioinformatics; Docker; Genomics; Pipeline; Reproducibility; Workflows.

PubMed Disclaimer

Conflict of interest statement

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Michael R. Crusoe, in his role as CWL Community Engineer, has had his salary supported in the past by grants from Seven Bridges Genomics, Inc to his employers. The other authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures

**Fig. 1**
Workflow A: whole genome/Exome NGS data analysis

**Fig. 2**
Workflow B: 16S rDNA diversity analysis

**Fig. 3**
Workflow C: genome wide-association studies

See this image and copyright information in PMC

Cited by

Genetic association and transferability for urinary albumin-creatinine ratio as a marker of kidney disease in four Sub-Saharan African populations and non-continental individuals of African ancestry.
Brandenburg JT, Chen WC, Boua PR, Govender MA, Agongo G, Micklesfield LK, Sorgho H, Tollman S, Asiki G, Mashinya F, Hazelhurst S, Morris AP, Fabian J, Ramsay M. Brandenburg JT, et al. Front Genet. 2024 May 15;15:1372042. doi: 10.3389/fgene.2024.1372042. eCollection 2024. Front Genet. 2024. PMID: 38812969 Free PMC article.
Genome-wide association study of population-standardised cognitive performance phenotypes in a rural South African community.
Soo CC, Brandenburg JT, Nebel A, Tollman S, Berkman L, Ramsay M, Choudhury A. Soo CC, et al. Commun Biol. 2023 Mar 27;6(1):328. doi: 10.1038/s42003-023-04636-1. Commun Biol. 2023. PMID: 36973338 Free PMC article.
Using a multiple-delivery-mode training approach to develop local capacity and infrastructure for advanced bioinformatics in Africa.
Ras V, Botha G, Aron S, Lennard K, Allali I, Claassen-Weitz S, Mwaikono KS, Kennedy D, Holmes JR, Rendon G, Panji S, Fields CJ, Mulder N. Ras V, et al. PLoS Comput Biol. 2021 Feb 25;17(2):e1008640. doi: 10.1371/journal.pcbi.1008640. eCollection 2021 Feb. PLoS Comput Biol. 2021. PMID: 33630830 Free PMC article.
Cancer treatment comes to age: from one-size-fits-all to next-generation sequencing (NGS) technologies.
Parvizpour S, Beyrampour-Basmenj H, Razmara J, Farhadi F, Shamsir MS. Parvizpour S, et al. Bioimpacts. 2024;14(4):29957. doi: 10.34172/bi.2023.29957. Epub 2023 Dec 23. Bioimpacts. 2024. PMID: 39104623 Free PMC article. Review.
DolphinNext: a distributed data processing platform for high throughput genomics.
Yukselen O, Turkyilmaz O, Ozturk AR, Garber M, Kucukural A. Yukselen O, et al. BMC Genomics. 2020 Apr 19;21(1):310. doi: 10.1186/s12864-020-6714-x. BMC Genomics. 2020. PMID: 32306927 Free PMC article.

See all "Cited by" articles

References

1. Kircher Martin, Kelso Janet. High-throughput DNA sequencing - concepts and limitations. BioEssays. 2010;32(6):524–536. doi: 10.1002/bies.200900181. - DOI - PubMed
1. Sandve Geir Kjetil, Nekrutenko Anton, Taylor James, Hovig Eivind. Ten Simple Rules for Reproducible Computational Research. PLoS Computational Biology. 2013;9(10):e1003285. doi: 10.1371/journal.pcbi.1003285. - DOI - PMC - PubMed
1. Schulz WadeL, Durant Thomas, Siddon AlexaJ, Torres Richard. Use of application containers and workflows for genomic data analysis. Journal of Pathology Informatics. 2016;7(1):53. doi: 10.4103/2153-3539.197197. - DOI - PMC - PubMed
1. Leipzig J. A review of bioinformatic pipeline frameworks. Brief Bioinform. 2017; 18(3):530–6. 10.1093/bib/bbw020. - PMC - PubMed
1. Liu Bo, Madduri Ravi K, Sotomayor Borja, Chard Kyle, Lacinski Lukasz, Dave Utpal J, Li Jianqiang, Liu Chunchen, Foster Ian T. Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses. Journal of Biomedical Informatics. 2014;49:119–133. doi: 10.1016/j.jbi.2014.01.005. - DOI - PMC - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources

[1] Kircher Martin, Kelso Janet. High-throughput DNA sequencing - concepts and limitations. BioEssays. 2010;32(6):524–536. doi: 10.1002/bies.200900181. - DOI - PubMed

[2] Kircher Martin, Kelso Janet. High-throughput DNA sequencing - concepts and limitations. BioEssays. 2010;32(6):524–536. doi: 10.1002/bies.200900181. - DOI - PubMed

[3] Sandve Geir Kjetil, Nekrutenko Anton, Taylor James, Hovig Eivind. Ten Simple Rules for Reproducible Computational Research. PLoS Computational Biology. 2013;9(10):e1003285. doi: 10.1371/journal.pcbi.1003285. - DOI - PMC - PubMed

[4] Sandve Geir Kjetil, Nekrutenko Anton, Taylor James, Hovig Eivind. Ten Simple Rules for Reproducible Computational Research. PLoS Computational Biology. 2013;9(10):e1003285. doi: 10.1371/journal.pcbi.1003285. - DOI - PMC - PubMed

[5] Schulz WadeL, Durant Thomas, Siddon AlexaJ, Torres Richard. Use of application containers and workflows for genomic data analysis. Journal of Pathology Informatics. 2016;7(1):53. doi: 10.4103/2153-3539.197197. - DOI - PMC - PubMed

[6] Schulz WadeL, Durant Thomas, Siddon AlexaJ, Torres Richard. Use of application containers and workflows for genomic data analysis. Journal of Pathology Informatics. 2016;7(1):53. doi: 10.4103/2153-3539.197197. - DOI - PMC - PubMed

[7] Leipzig J. A review of bioinformatic pipeline frameworks. Brief Bioinform. 2017; 18(3):530–6. 10.1093/bib/bbw020. - PMC - PubMed

[8] Leipzig J. A review of bioinformatic pipeline frameworks. Brief Bioinform. 2017; 18(3):530–6. 10.1093/bib/bbw020. - PMC - PubMed

[9] Liu Bo, Madduri Ravi K, Sotomayor Borja, Chard Kyle, Lacinski Lukasz, Dave Utpal J, Li Jianqiang, Liu Chunchen, Foster Ian T. Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses. Journal of Biomedical Informatics. 2014;49:119–133. doi: 10.1016/j.jbi.2014.01.005. - DOI - PMC - PubMed

[10] Liu Bo, Madduri Ravi K, Sotomayor Borja, Chard Kyle, Lacinski Lukasz, Dave Utpal J, Li Jianqiang, Liu Chunchen, Foster Ian T. Cloud-based bioinformatics workflow platform for large-scale next-generation sequencing analyses. Journal of Biomedical Informatics. 2014;49:119–133. doi: 10.1016/j.jbi.2014.01.005. - DOI - PMC - PubMed

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed