Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jan 1:2019:baz096.
doi: 10.1093/database/baz096.

Benchmarking database systems for Genomic Selection implementation

Yaw Nti-Addae  1 , Dave Matthews  2 , Victor Jun Ulat  3 , Raza Syed  1 , Guilhem Sempéré  4 , Adrien Pétel  5 , Jon Renner  6 , Pierre Larmande  7 , Valentin Guignon  8 , Elizabeth Jones  1 , Kelly Robbins  9
Affiliations

Benchmarking database systems for Genomic Selection implementation

Yaw Nti-Addae et al. Database (Oxford). .

Abstract

Motivation: With high-throughput genotyping systems now available, it has become feasible to fully integrate genotyping information into breeding programs. To make use of this information effectively requires DNA extraction facilities and marker production facilities that can efficiently deploy the desired set of markers across samples with a rapid turnaround time that allows for selection before crosses needed to be made. In reality, breeders often have a short window of time to make decisions by the time they are able to collect all their phenotyping data and receive corresponding genotyping data. This presents a challenge to organize information and utilize it in downstream analyses to support decisions made by breeders. In order to implement genomic selection routinely as part of breeding programs, one would need an efficient genotyping data storage system. We selected and benchmarked six popular open-source data storage systems, including relational database management and columnar storage systems.

Results: We found that data extract times are greatly influenced by the orientation in which genotype data is stored in a system. HDF5 consistently performed best, in part because it can more efficiently work with both orientations of the allele matrix.

Availability: http://gobiin1.bti.cornell.edu:6083/projects/GBM/repos/benchmarking/browse.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Different orientation of genotyping data. (a) markers in rows and samples in columns, whereas (b) shows markers in columns and samples in rows.
Figure 2
Figure 2
Load times for database systems.
Figure 3
Figure 3
Times for extracting increasing number of markers across all samples. (a) Times for extracting a contiguous set of markers for all samples. Times for MariaDB are excluded because they exceed 25 hours, and times for MongoDB and MonetDB were essentially identical. (b) Times for extracting a random set of markers for all samples. Times for MariaDB and MonetDB are excluded since they exceed 25 hours. (c) A zoom-in at extract times up to 500,000 random markers to show if there is significant difference between HDF5, MongoDB and PostgreSQL.
Figure 4
Figure 4
Times for extracting increasing number of samples across all markers. (a) Times for extracting contiguous set of samples for all 32 million markers. (b) Times for extracting random set of samples for all 32 million markers. Times for MariaDB and PostgreSQL are excluded in both (a) and (b) because their queries exceeded 20 hours.
Figure 5
Figure 5
Times for extracting increasing cross-section of samples and 1 million markers. Time for extracting a block contiguous number of samples and 1 million contiguous markers. Extract times for MariaDB are excluded since they exceeded 25 hours. (b) Times for extracting random set of samples across 1 million random markers. Extract times for MariaDB and MonetDB are excluded because their queries exceeded 25 hours.
See this image and copyright information in PMC

References

    1. Meuwissen T.H.E., Hayes B.J. and Goddard M.E. (2001) Prediction of Total Genetic Value Using Genome-Wide Dense Marker Maps, Genetics, 157, 1819 LP-1829. - PMC - PubMed
    1. Hickey J.M., Chiurugwi T., Mackay I. et al. (2017) Genomic prediction unifies animal and plant breeding programs to form platforms for biological discovery. Nat. Genet., 49, 1297. - PubMed
    1. Lin Z., Hayes B.J. and Daetwyler H.D. (2014) Genomic selection in crops, trees and forages: a review, Crop Pasture Sci., 65, 1177–1191.
    1. Wang S., et al. (2014) High dimensional biological data retrieval optimization with NoSQL technology, BMC genomics, 15, S3. - PMC - PubMed
    1. Röhm U. and Blakeley J. (2009) Data management for high-throughput genomics, arXiv Prepr. arXiv0909.1764.

Publication types