Detection of Pathogenic Variants With Germline Genetic Testing Using Deep Learning vs Standard Methods in Patients With Prostate Cancer and Melanoma

Abstract

Importance: Less than 10% of patients with cancer have detectable pathogenic germline alterations, which may be partially due to incomplete pathogenic variant detection.

Objective: To evaluate if deep learning approaches identify more germline pathogenic variants in patients with cancer.

Design, setting, and participants: A cross-sectional study of a standard germline detection method and a deep learning method in 2 convenience cohorts with prostate cancer and melanoma enrolled in the US and Europe between 2010 and 2017. The final date of clinical data collection was December 2017.

Exposures: Germline variant detection using standard or deep learning methods.

Main outcomes and measures: The primary outcomes included pathogenic variant detection performance in 118 cancer-predisposition genes estimated as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The secondary outcomes were pathogenic variant detection performance in 59 genes deemed actionable by the American College of Medical Genetics and Genomics (ACMG) and 5197 clinically relevant mendelian genes. True sensitivity and true specificity could not be calculated due to lack of a criterion reference standard, but were estimated as the proportion of true-positive variants and true-negative variants, respectively, identified by each method in a reference variant set that consisted of all variants judged to be valid from either approach.

Results: The prostate cancer cohort included 1072 men (mean [SD] age at diagnosis, 63.7 [7.9] years; 857 [79.9%] with European ancestry) and the melanoma cohort included 1295 patients (mean [SD] age at diagnosis, 59.8 [15.6] years; 488 [37.7%] women; 1060 [81.9%] with European ancestry). The deep learning method identified more patients with pathogenic variants in cancer-predisposition genes than the standard method (prostate cancer: 198 vs 182; melanoma: 93 vs 74); sensitivity (prostate cancer: 94.7% vs 87.1% [difference, 7.6%; 95% CI, 2.2% to 13.1%]; melanoma: 74.4% vs 59.2% [difference, 15.2%; 95% CI, 3.7% to 26.7%]), specificity (prostate cancer: 64.0% vs 36.0% [difference, 28.0%; 95% CI, 1.4% to 54.6%]; melanoma: 63.4% vs 36.6% [difference, 26.8%; 95% CI, 17.6% to 35.9%]), PPV (prostate cancer: 95.7% vs 91.9% [difference, 3.8%; 95% CI, -1.0% to 8.4%]; melanoma: 54.4% vs 35.4% [difference, 19.0%; 95% CI, 9.1% to 28.9%]), and NPV (prostate cancer: 59.3% vs 25.0% [difference, 34.3%; 95% CI, 10.9% to 57.6%]; melanoma: 80.8% vs 60.5% [difference, 20.3%; 95% CI, 10.0% to 30.7%]). For the ACMG genes, the sensitivity of the 2 methods was not significantly different in the prostate cancer cohort (94.9% vs 90.6% [difference, 4.3%; 95% CI, -2.3% to 10.9%]), but the deep learning method had a higher sensitivity in the melanoma cohort (71.6% vs 53.7% [difference, 17.9%; 95% CI, 1.82% to 34.0%]). The deep learning method had higher sensitivity in the mendelian genes (prostate cancer: 99.7% vs 95.1% [difference, 4.6%; 95% CI, 3.0% to 6.3%]; melanoma: 91.7% vs 86.2% [difference, 5.5%; 95% CI, 2.2% to 8.8%]).

Conclusions and relevance: Among a convenience sample of 2 independent cohorts of patients with prostate cancer and melanoma, germline genetic testing using deep learning, compared with the current standard genetic testing method, was associated with higher sensitivity and specificity for detection of pathogenic variants. Further research is needed to understand the relevance of these findings with regard to clinical outcomes.

Figure 1.. Overview of the Study Design and the Germline Variant Detection Methods Used

Germline whole-exome sequencing data of 2 independent cohorts were analyzed using a deep learning variant detection approach and the current standard germline variant detection method to assess the sensitivity, specificity, positive predictive value, and negative predictive value of detecting pathogenic variants in 118 cancer-predisposition genes as well as in 59 American College of Medical Genetics and Genomics (ACMG) genes. In addition, an exome-wide analysis of putative loss-of-function (pLOF) variants in 5197 Online Mendelian Inheritance in Man (OMIM) genes and in 12 clinically oriented multigene panels was conducted in 286 patients with prostate cancer and in 1295 patients with melanoma whose tumors were available for independent variant validation. For each analysis, the performance of the deep learning method and the standard method was compared with the performance obtained when both methods are concurrently used (additional information appears in the Methods section). The performance of the standard and deep learning methods to detect pathogenic variants in the cancer-predisposition and ACMG gene sets was tested on all patients with prostate cancer and melanoma (n = 2367). However, the performance of the 2 methods in the OMIM gene set and the multigene panels was tested on 286 patients with prostate cancer and 1295 patients with melanoma (n = 1581) whose tumor sequencing data were available for computational validation of the identified pLOF variants. ^aRepresents the frequency of the allele (also called variant) in the general population. ^bGenetic variants that are likely to severely disrupt the protein function by truncating the gene transcript. Examples of pLOF variants include frameshift, stop-codon, and canonical splice-site variants.

Figure 2.. Examples of Germline Pathogenic Variants That Were Exclusively Identified by the Deep Learning or Standard Variant Detection Methods

Thirty-six germline pathogenic variants were identified by the deep learning method, 75% of which were judged to be valid or true-positive (A), whereas 27 variants were identified by the standard method, 40.7% of which were judged to be valid (B). A similar analysis of 1295 patients with melanoma showed a similarly higher detection rate with the deep learning method (eFigure 4). Examples show pathogenic variants in the cancer-predisposition gene set (C-E) and the clinically actionable American College of Medical Genetics and Genomics gene set (F-H) that were only identified by the deep learning method and judged to be valid on manual assessment.

Figure 3.. Evaluation of Model Performance and Normalized Confusion Matrices of the Standard and Deep Learning Methods

Manually assessed pathogenic variants in 151 cancer-predisposition and American College of Medical Genetics and Genomics genes were used to calculate the receiver operating characteristic curve using a set of thresholds for the quality scores. For each quality threshold, the standard and deep learning models predicted if the variant being assessed is real or artifactual. These predictions were then compared with the results of the manual validation of these pathogenic variants (judged to be valid or not) and true-positive, true-negative, false-positive, and false-negative rates were calculated. Analysis of prostate cancer cohort (n = 1072) (A). Normalized confusion matrices (B and C), representing the fractions of the manually validated true-positive and false-positive variants (ie, the reference variant set; y-axis) that were correctly identified as such by the standard method and the deep learning method (x-axis). For each matrix, the left-upper and right-lower squares represent the degree of agreement while the right-upper and left-lower squares represent the degree of mismatching between the results of manual validation and the method-based variant assessment. The intensity of the square shading represents the metric within the square, which is the ratio of the number of variants in each category to the total number of true-positive or true-negative in the reference variant set (ie, the manual validation set). Compared with the standard method, there was more agreement between the variant assessment results of the deep learning method and the manual validation results in the prostate cancer cohort (n = 1072). Analysis of the melanoma cohort (n = 1295) (D). Similarly, there was more agreement between the deep learning–based variant assessment and the manual evaluation of these variants in the raw genomic data of the melanoma cohort (E and F).