Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions

Abstract

Background: Targeted sequencing using oncopanels requires comprehensive assessments of accuracy and detection sensitivity to ensure analytical validity. By employing reference materials characterized by the U.S. Food and Drug Administration-led SEquence Quality Control project phase2 (SEQC2) effort, we perform a cross-platform multi-lab evaluation of eight Pan-Cancer panels to assess best practices for oncopanel sequencing.

Results: All panels demonstrate high sensitivity across targeted high-confidence coding regions and variant types for the variants previously verified to have variant allele frequency (VAF) in the 5-20% range. Sensitivity is reduced by utilizing VAF thresholds due to inherent variability in VAF measurements. Enforcing a VAF threshold for reporting has a positive impact on reducing false positive calls. Importantly, the false positive rate is found to be significantly higher outside the high-confidence coding regions, resulting in lower reproducibility. Thus, region restriction and VAF thresholds lead to low relative technical variability in estimating promising biomarkers and tumor mutational burden.

Conclusion: This comprehensive study provides actionable guidelines for oncopanel sequencing and clear evidence that supports a simplified approach to assess the analytical performance of oncopanels. It will facilitate the rapid implementation, validation, and quality control of oncopanels in clinical use.

Keywords: Analytical performance; Molecular diagnostics; Oncopanel sequencing; Precision medicine; Reproducibility; Target enrichment.

Fig. 1

Comprehensive study design for assessing analytical performance of multiple pan-cancer targeted sequencing technologies. a Four samples were tested on 8 pan-cancer panels with at least 3 different test laboratories for each panel. b Basic information of 8 pan-cancer panels is listed in the embedded table (see Additional file 1: Table S1 for detailed information). *All participating panels are for research use only. ^†QGN’s UMI-aware variant caller is able to call variants with VAF as low as 0.5%. c Each sample had 4 library replicates at each test laboratory. After sequencing, panel-specific variant calling was performed by each panel vendor. d Variant calling results were submitted for performance analysis including sensitivity, false positive call rate, and reproducibility

Fig. 2

Reproducibility and sensitivity across VAF ranges for SNVs in the consensus targeted regions. a Table listed the number of known variants in each VAF range (left number), sensitivity (right number) for all 8 panels across all samples tested. For the panels with a built-in VAF threshold, “N/A” is listed if the VAF low bound is much lower than the panel provider’s chosen VAF threshold. The VAF threshold is 2.6% for ILM, 2.0% for IDT, 2.5% for ROC, and 2.5% for TFS, respectively. b Average false positive SNV calls per million across various VAF cutoffs. Jittering was applied to avoid overlapping. c Cross-lab and intra-lab reproducibility (in Phred scale) for variant calls with VAF between 2.5 and 20%

Fig. 3

Impact of VAF cutoff, variant type, and genomic region on sensitivity (in Phred scale). a Violin distribution plots of estimated sensitivity for each panel in all sample A and C libraries for known SNVs (in blue on the left side) and other variants (small indels or MNVs, in green on the adjacent right side) with VAF between 2.5 and 20%. Total numbers of small indels and MNVs are listed under the corresponding violin plot. b Artificial VAF filters reduce sensitivity for known positives with VAF between 2.5 and 5% due to the variable VAF measurements. c High concordance of sensitivity in and outside of CTR (more specifically, in HC_CR beyond CTR) for known positives with VAF between 2.5 and 20%. Jittering was applied to one dot at the top right corner to avoid overlapping

Fig. 4

Impact of CTR region on FP rate and reproducibility. a FP rate in and outside of the CTR using two different methods (B_low and C_only) at three different VAF cutoffs, 1%, 2.5%, and 5%. C_only was not applied to TFS as sample C was not tested on TFS. FP rates are plotted in squared root scale. b Estimated FP rate within the CTR averaged over three methods at different VAF cutoffs. c Cross-lab reproducibility (in Phred scale) in samples A and C within and outside of the CTR

Fig. 5

Coefficient of variation (CV) of TMB. a Technical run-to-run variance of TMB with VAF cutoff above 2.5% was estimated for six panels at different TMB levels. A power-law curve (dashed line) is fitted for each panel. b Technical run-to-run variance of TMB with VAF cutoff above 5%. c The intrinsic CV is plotted with the equation (embedded, see “Methods” for detail) for each panel based on their panel size. The curve (dashed) for size of 1 Mb is also plotted as a reference. d The overall CV is plotted combining technical and intrinsic CV. The solid fitting power-law curve is for TMB (VAF > 2.5%), and the dashed curve is for TMB (VAF > 5%)