Assessing the performance of in silico methods for predicting the pathogenicity of variants in the gene CHEK2, among Hispanic females with breast cancer

Abstract

The availability of disease-specific genomic data is critical for developing new computational methods that predict the pathogenicity of human variants and advance the field of precision medicine. However, the lack of gold standards to properly train and benchmark such methods is one of the greatest challenges in the field. In response to this challenge, the scientific community is invited to participate in the Critical Assessment for Genome Interpretation (CAGI), where unpublished disease variants are available for classification by in silico methods. As part of the CAGI-5 challenge, we evaluated the performance of 18 submissions and three additional methods in predicting the pathogenicity of single nucleotide variants (SNVs) in checkpoint kinase 2 (CHEK2) for cases of breast cancer in Hispanic females. As part of the assessment, the efficacy of the analysis method and the setup of the challenge were also considered. The results indicated that though the challenge could benefit from additional participant data, the combined generalized linear model analysis and odds of pathogenicity analysis provided a framework to evaluate the methods submitted for SNV pathogenicity identification and for comparison to other available methods. The outcome of this challenge and the approaches used can help guide further advancements in identifying SNV-disease relationships.

Keywords: CAGI; CHEK2; Hispanic women; SNV; breast cancer.

Figure 1.

Distribution of patients in case control category per SNV position

Figure 2

Representation of submission P(case). Green indicates values close to 0-benign and red indicates values close to 1-pathogenic.

Figure 3

Summary of Odds of Pathogenicity results for all the submissions and reference methods used in the assessment. Dot size is proportional to the number of positions in each of the benign (green), indeterminate (cyan) and pathogenic (red) categories. Graph background color shows the ACMG odds of pathogenicity ranges use as guide (green-benign, blue-indeterminate, orange-pathogenic)

Figure 4

Representative positions for cancerous, neutral and protective variants. Red dots represent a cancerous position, black dots represent a neutral position and green dots represent protective variants. The blue line indicates the 0.5 neutral p-value.