Comparative analysis of functional assay evidence use by ClinGen Variant Curation Expert Panels

Abstract

Background: The 2015 American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) guidelines for clinical sequence variant interpretation state that "well-established" functional studies can be used as evidence in variant classification. These guidelines articulated key attributes of functional data, including that assays should reflect the biological environment and be analytically sound; however, details of how to evaluate these attributes were left to expert judgment. The Clinical Genome Resource (ClinGen) designates Variant Curation Expert Panels (VCEPs) in specific disease areas to make gene-centric specifications to the ACMG/AMP guidelines, including more specific definitions of appropriate functional assays. We set out to evaluate the existing VCEP guidelines for functional assays.

Methods: We evaluated the functional criteria (PS3/BS3) of six VCEPs (CDH1, Hearing Loss, Inherited Cardiomyopathy-MYH7, PAH, PTEN, RASopathy). We then established criteria for evaluating functional studies based on disease mechanism, general class of assay, and the characteristics of specific assay instances described in the primary literature. Using these criteria, we extensively curated assay instances cited by each VCEP in their pilot variant classification to analyze VCEP recommendations and their use in the interpretation of functional studies.

Results: Unsurprisingly, our analysis highlighted the breadth of VCEP-approved assays, reflecting the diversity of disease mechanisms among VCEPs. We also noted substantial variability between VCEPs in the method used to select these assays and in the approach used to specify strength modifications, as well as differences in suggested validation parameters. Importantly, we observed discrepancies between the parameters VCEPs specified as required for approved assay instances and the fulfillment of these requirements in the individual assays cited in pilot variant interpretation.

Conclusions: Interpretation of the intricacies of functional assays often requires expert-level knowledge of the gene and disease, and current VCEP recommendations for functional assay evidence are a useful tool to improve the accessibility of functional data by providing a starting point for curators to identify approved functional assays and key metrics. However, our analysis suggests that further guidance is needed to standardize this process and ensure consistency in the application of functional evidence.

Keywords: Clinical exome sequencing; Clinical genome resource; Clinical genome sequencing; Functional assays; Variant interpretation.

Fig. 1

Use of the PS3/BS3 criteria in Variant Curation Expert Panel (VCEP) pilot variant classification. a Comparison of PS3 criterion application at any strength level (purple) and BS3 criterion application at any strength level (orange) in the pilot variant classification of each VCEP. b Comparison of PS3 criterion application at any strength level (purple) and BS3 criterion application at any strength level (orange) to variants ultimately classified as variants of uncertain significance (VUS) in the pilot variant classification of each VCEP. c Comparison of the final classification (P, LP, VUS, LB, or B) of pilot variants with PS3/BS3 criteria (at any strength level). The CDH1 VCEP and the Inherited Cardiomyopathy-MYH7 VCEP did not use BS3 evidence in the interpretation of any pilot variants

Fig. 2

Representative findings from the curation of specific instances of CDH1 functional assays. We assessed the following methods and validation parameters of aggregation/invasion, wound closure, and proximity ligation assays: experimental material, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP), see Additional file 2: Tables S1 and S2 for the full primary literature curation results

Fig. 3

Representative findings from the curation of specific instances of GJB2 functional assays. We assessed the following methods and validation parameters of electrical coupling and dye transfer assays: experimental material, controls, replication, and output. ^†Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S4 and S5 for the full primary literature curation results

Fig. 4

Representative findings from the curation of specific instances of MYH7 functional assays. We assessed the following methods and validation parameters of variant-specific knock-in mouse models, ATPase activity, and motility assays: experimental material, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP). ^†Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S6 and S7 for the primary literature curation results

Fig. 5

Representative findings from the curation of specific instances of PAH functional assays. We assessed the following methods and validation parameters of enzyme activity and protein folding/stability assays: experimental material, cofactor, detection method, controls, replication, and output. *Assays not approved by the Variant Curation Expert Panel (VCEP). ^†Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification. TLC, thin-layer chromatography; HPLC-Fluorescence, high-performance liquid chromatography coupled with fluorescence detection; LC-MS, liquid chromatography coupled with mass spectrometry, see Additional file 2: Tables S8 and S9 for the full primary literature curation results

Fig. 6

Representative findings from the curation of specific instances of PTEN functional assays. We assessed the following methods and validation parameters of phosphatase activity, PTEN level, pAKT level, protein localization, cell proliferation, and cell migration assays: experimental material, controls, replication, and output. ^†Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification, see Additional file 2: Tables S10 and S11 for the full primary literature curation results

Fig. 7

Representative findings from the curation of specific instances of RASopathy functional assays. We assessed the following methods and validation parameters of MAP 2K1/2 and ERK1/2 phosphorylation assays: cell treatment, controls, replication, and output. ^†Specific instance of assay cited by the Variant Curation Expert Panel (VCEP) as evidence for the PS3/BS3 criteria in pilot variant classification. ^‡Specific instance of assay cited by the VCEP as evidence for the PS3/BS3 criteria in pilot variant classification for variants in multiple genes: PMID 18413255 cited as evidence for the PS3/BS3 criteria for variants in BRAF and MAP 2K1/2; PMID 16439621 cited as evidence for PS3/BS3 for variants in KRAS and MAP 2 K1/2; and PMID 21784453 cited as evidence for PS3/BS3 for variants in RAF1 and SOS1, see Additional file 2: Tables S12 and S13 for the full primary literature curation results