Analytical Validation of the Next-Generation Sequencing Assay for a Nationwide Signal-Finding Clinical Trial: Molecular Analysis for Therapy Choice Clinical Trial

Abstract

The National Cancer Institute-Molecular Analysis for Therapy Choice (NCI-MATCH) trial is a national signal-finding precision medicine study that relies on genomic assays to screen and enroll patients with relapsed or refractory cancer after standard treatments. We report the analytical validation processes for the next-generation sequencing (NGS) assay that was tailored for regulatory compliant use in the trial. The Oncomine Cancer Panel assay and the Personal Genome Machine were used in four networked laboratories accredited for the Clinical Laboratory Improvement Amendments. Using formalin-fixed paraffin-embedded clinical specimens and cell lines, we found that the assay achieved overall sensitivity of 96.98% for 265 known mutations and 99.99% specificity. High reproducibility in detecting all reportable variants was observed, with a 99.99% mean interoperator pairwise concordance across the four laboratories. The limit of detection for each variant type was 2.8% for single-nucleotide variants, 10.5% for insertion/deletions, 6.8% for large insertion/deletions (gap ≥4 bp), and four copies for gene amplification. The assay system from biopsy collection through reporting was tested and found to be fully fit for purpose. Our results indicate that the NCI-MATCH NGS assay met the criteria for the intended clinical use and that high reproducibility of a complex NGS assay is achievable across multiple clinical laboratories. Our validation approaches can serve as a template for development and validation of other NGS assays for precision medicine.

Figure 1

Workflows and components in the NCI-MATCH assay system. Four core needle specimens biopsied from a patient are shipped to central pathology laboratory at MD Anderson Cancer Center (MDACC). Tissue specimens are registered, and the preanalytic processes include formalin fixation and paraffin embedding (FFPE), hematoxylin and eosin staining, and tumor content assessment. The content of tumor cellularity of a specimen is required to be ≥70%, and enrichment by manual microdissections is performed on all specimens to attempt to achieve >70% tumor cellularity before nucleic acid (NA) extraction. A PTEN immunohistochemistry (IHC) stain is also performed as an inclusion biomarker for one treatment arm in the NCI-MATCH study. All residual tissue and nucleic acid samples are achieved in the ECOG-ACRIN Central Biorepository and Pathology Facility at the MDACC. One patient's genomic DNA (gDNA) and cDNA (reverse transcribed from total RNA) are shipped to one of four clinical laboratories to perform the NCI-MATCH next-generation sequencing (NGS) assay. The NGS data are analyzed by Ion Reporter (Torrent Suite version 4.4.2 Oncomine workflow) and the variant calling format (VCF) file and binary alignment map (BAM) file are automatically uploaded to MATCHBox, the centralized data processing system. MATCHBox identifies the actionable mutations of interest and assigns the treatment. The variant report and treatment assignment are reviewed by a group of experts composed of bioinformaticans, molecular biologists, statisticians, oncologists, and pathologists before sign off of the final report. The final results are sent to the enrolling physicians and patients and to a clinical medical database (DB) for long-term storage. The overall turnaround time from biopsy to final report is intended to be between 11 and 14 days. FNLCR, Frederick National Laboratory for Cancer Research; MGH, Massachusetts General Hospital; YSM, Yale University School of Medicine.

Figure 2

Tissue type has no effect on library yield. Library yields are plotted against the tissue types from which DNA samples (A) and RNA samples (B) were extracted. Each dot represents a sample, and each color code represents the laboratory site where the NCI-MATCH NGS assay was performed. The red dashed lines represent a minimal yield of 20 pmol/L required for template preparation. FNLCR, Frederick National Laboratory for Cancer Research; MDACC, MD Anderson Cancer Center; MGH, Massachusetts General Hospital; YSM, Yale University School of Medicine.

Figure 3

Reproducibility assessment across 16 technical replicates in four network laboratories. A and B: Reproducibility of 45 positive variants consisting of 38 variants detected in 9 DNA samples (A) and 7 fusion variants in 7 RNA samples (B) that were called at least once in 16 technical replicates across 4 network laboratories. Each row represents a variant, and the type of variant, sample, gene, Catalogue of Somatic Mutations in Cancer(COSMIC) identification or position (hg19) for variants without COSMIC identification are labeled. Each column represents a replicate; the laboratory and operator and replicate are indicated. The color code represents a variant call by the NCI-MATCH NGS assay (gray, no call; brown, a call). A: Eight DNA variants discordantly called (top) include 7 variants (6 indels and 1 SNV) detected once and 1 variant (Indel) detected only twice in 16 technical replicates. To simplify the presentation, replicates generated from OP2 and OP3 at the Yale University School of Medicine (YSM) are grouped and indicated as OP2. C and D: Allele frequencies of 27 variants (C) and copy numbers of 3 variants (D) were concordantly detected in 16 technical replicates and are plotted as box plot. Each row represents a variant, and the allele frequency copy number detected in 16 replicates are grouped by laboratory and color-coded as indicated in the legends. FNLCR, Frederick National Laboratory for Cancer Research; indel, insertion and deletion; MDACC, MD Anderson Cancer Center; MGH, Massachusetts General Hospital; OP, operator.