Benchmarking accelerated next-generation sequencing analysis pipelines

Abstract

Motivation: Industry-standard central processing unit (CPU)-based next-generation sequencing (NGS) analysis tools have led to longer runtimes, affecting their utility in time-sensitive clinical practices and population-scale research studies. To address this, researchers have developed accelerated NGS platforms like DRAGEN and Parabricks, which have significantly reduced runtimes-from days to hours. However, these studies have evaluated accelerated platforms independently without sufficiently assessing computational resource usage or thoroughly investigating speedup scalability, a gap our study is designed to address.

Results: Corroborating previous studies, accelerated pipelines demonstrated shorter runtimes than CPU-only approaches, with Parabricks-H100 demonstrating the highest speedups, followed by DRAGEN. In mapping, DRAGEN outperformed Parabricks (L4 and A100) and matched H100 speedups. Parabricks (A100 and H100) variant calling demonstrated higher speedups than DRAGEN. Moreover, DRAGEN and Parabricks-H100 mapping showed positive trends in the coverage-based scalability analysis, while other configurations failed to scale effectively. Our profiler analysis provided new insights into the relationships between Parabricks' performances and resource usage patterns, revealing its potential for further improvements. Our findings and cost comparison help researchers select accelerated platforms based on coverage needs, timeframes, and budget, while suggesting optimization strategies.

Availability and implementation: Datasets are described in the 'Data availability' section. Our NGS pipelines are available at https://github.com/NAICNO/accelerated_genomics.

Figure 1.

Overview of the three NGS pipelines—DRAGEN, CPU-only, and Parabricks. The workflow diagram compares three NGS pipelines, showing the input and output at each processing step.

Figure 2.

Runtime and speedup comparison. (A–C) Runtime comparisons (in minutes) across accelerated platforms for (A) read mapping stage, (B) variant calling HC, and (C) variant calling DV processes. Accelerated platforms include Parabricks on L4, A100, and H100 GPUs and DRAGEN. CPU-only pipeline data omitted to highlight runtime differences between accelerated pipelines. (D–F) Speedup factors of accelerated (D) read mapping stage, (E) variant calling HC and (F) variant calling DV, calculated relative to CPU-only baseline (Speedup = baseline runtime/accelerated runtime).

Figure 3.

Resource usage patterns across coverage and GPU platforms. (A–C) L4 and H100 GPU (cards) usage comparison between low- and high-coverage samples during (A) read mapping stage, (B) variant calling HC, and (C) variant calling DV processes. (D–F) L4 and H100 GPU memory usage comparison for low- and high-coverage samples during (D) read mapping stage, variant calling (E) HC, and (F) variant calling DV processes.

Figure 4.

Disk I/O usage of Parabricks pipeline (on H100 GPUs). (A) Disk I/O rate distribution across three processes of the Parabricks pipeline; (B) same distribution in log10 scale to highlight read/write activity across the Parabricks pipeline.

Figure 5.

Performance evaluation and concordance analysis. (A, B) Precision, recall, and F1 scores from performance evaluation analysis of (A) SNVs and (B) indels. Performance evaluation analysis with the hap.py tool used VCF files of accelerated pipelines as query and the gold-standard high-confidence VCF files as the ground truth; (C, D) precision, recall, and F1 scores from concordance analysis of (C) SNVs and (D) indels. Concordance analysis with the hap.py tool used VCF files of accelerated pipelines as query and the CPU-only best-practice output VCF files as the ground truth.