Validating data from multiplex assays of variant effect: A CanVIG-UK national survey of NHS clinical scientists

Abstract

Advances in technology have made it possible for multiplex assays of variant effect (MAVEs) to systematically generate functional data for thousands of genetic variants. Robust clinical validation and accessible online resources for MAVE data have previously been identified as barriers to the clinical adoption of new MAVEs. We delivered a survey during the November 2024 Cancer Variant Interpretation Group UK (CanVIG-UK) meeting comprising National Health Service (NHS) clinical scientists and clinical geneticists and received 46 responses from individuals regularly performing variant classification for diagnostic reporting. Only 35% reported they would accept clinical validation of the MAVE provided by the authors who conducted the assay; 20% reported they would attempt clinical validation themselves, and 61% would await clinical validation by a trusted central body. 72% reported they would use MAVE data ahead of a formal peer-reviewed publication if reviewed and clinically validated by a trusted central body. When scoring central bodies on a scale of 1-5 for confidence in their review and validation of MAVEs, CanVIG-UK (median = 5), variant curation expert panels (VCEPs; median = 5), and ClinGen SVI Functional Working Group (median = 4) all scored highly. Participants supported making variant-level data accessible via a relevant web resource (although the majority of participants expressed that additional assay-level or variant-level information would have a low likelihood of altering validation scores provided by a trusted central body). These findings, from a comparatively homogeneous clinical diagnostic group operating in a resource-constrained healthcare setting, indicate that clinical application of new MAVEs for variant classification will be delayed unless robust clinical validations are performed by a trusted central body and made readily accessible.

Keywords: ACMG/AMP variant classification framework; Brnich-style validation; CanVIG-UK; MAVE data; PS3/BS3 scores; clinical variant classification; variant truth sets.

Figure 1

Results from survey questions pertaining to local clinical validation. n = Number of participants. A: Number of participants reporting if they have or have not ever attempted a Brnich-style validation themselves (n=46). B: Number of participants who would or would not undertake Brnich-style validation for a new MAVE of utility for a gene (n=46). C: Reasons why participants would not attempt a Brnich-style validation locally (n=37). Participants could select one or more of the given options for this question.

Figure 2

Results from survey questions on centralised clinical validation. n = Number of participants. A: Opinions on using assay data where validation performed only by the MAVE assay authors (n=46). B: Opinions on using pre-print assay data validated by a ‘trusted central body’ (n=46). C: Confidence of participants who would use validated pre-print assay data (n=33) which has been validated by each of the listed central bodies, scored from 1-5 (1=very unconfident, 5=very confident). Mean average and median scores are displayed; mean scores are represented by a red point for each group. One participant did not provide a score for the Atlas of Variant Effect Alliance group (n=32).

Figure 3

Participant scoring of the likelihood that assay-level information (type of cell line or type of assay) and/or variant-level information (number and consistency of replicates, and absolute functional score) will influence the PS3/BS3 evidence strength applied, scored from 1-5 (n=46, 1=low likelihood, 5=high likelihood). Mean average and median scores are displayed; mean scores are represented by a red point for each group.