Predicting the impact of rare variants on RNA splicing in CAGI6

Jenny Lord¹, Carolina Jaramillo Oquendo¹, Htoo A Wai¹, Andrew G L Douglas^{1

2}, David J Bunyan^{1

3}, Yaqiong Wang⁴, Zhiqiang Hu⁵, Zishuo Zeng⁶, Daniel Danis⁷, Panagiotis Katsonis⁸, Amanda Williams⁸, Olivier Lichtarge⁸, Yuchen Chang^{9

10}, Richard D Bagnall^{9

10}, Stephen M Mount¹¹, Brynja Matthiasardottir^{12

13}, Chiaofeng Lin¹⁴, Thomas van Overeem Hansen^{15

16}, Raphael Leman^{17

18}, Alexandra Martins¹⁹, Claude Houdayer^{19

20}, Sophie Krieger^{17

18}, Constantina Bakolitsa⁵, Yisu Peng²¹, Akash Kamandula²¹, Predrag Radivojac²¹, Diana Baralle^{22

23}

Affiliations

¹ Human Development and Health, Faculty of Medicine, University of Southampton, Southampton, UK.
² Oxford Centre for Genomic Medicine, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
³ Wessex Regional Genetics Laboratory, Salisbury District Hospital, Salisbury, UK.
⁴ Center for Molecular Medicine, Children's Hospital of Fudan University, National Children's Medical Center, Shanghai, 201102, China.
⁵ University of California, Berkeley, Berkeley, CA, 94720, USA.
⁶ Department of Biochemistry and Microbiology, Rutgers University, New Brunswick, NJ, 08873, USA.
⁷ The Jackson Laboratory for Genomic Medicine, 10 Discovery Drive, Farmington, CT, 06032, USA.
⁸ Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX, 77030, USA.
⁹ Agnes Ginges Centre for Molecular Cardiology at Centenary Institute, University of Sydney, Sydney, Australia.
¹⁰ Faculty of Medicine and Health, University of Sydney, Sydney, Australia.
¹¹ Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, MD, USA.
¹² Graduate Program in Biological Sciences and Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, MD, USA.
¹³ Inflammatory Disease Section, National Human Genome Research Institute, Bethesda, MD, USA.
¹⁴ DNAnexus, Mountain View, CA, 94040, USA.
¹⁵ Department of Clinical Genetics, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark.
¹⁶ Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
¹⁷ Laboratoire de Biologie et Génétique du Cancer, Centre François Baclesse, Caen, France.
¹⁸ Inserm U1245, Cancer Brain and Genomics, Normandie Université, UNICAEN, FHU G4 génomique, Rouen, France.
¹⁹ Inserm U1245, Cancer Brain and Genomics, Normandie Université, UNIROUEN, FHU G4 génomique, Rouen, France.
²⁰ Department of Genetics, Univ Rouen Normandie, INSERM U1245, FHU-G4 Génomique and CHU Rouen, 76000, Rouen, France.
²¹ Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02115, USA.
²² Human Development and Health, Faculty of Medicine, University of Southampton, Southampton, UK. d.baralle@soton.ac.uk.
²³ Wessex Clinical Genetics Service, University Hospital Southampton NHS Foundation Trust, Southampton, UK. d.baralle@soton.ac.uk.

PMID: 38170232
PMCID: PMC11976748
DOI: 10.1007/s00439-023-02624-3

Predicting the impact of rare variants on RNA splicing in CAGI6

Jenny Lord et al. Hum Genet. 2025 Mar.

. 2025 Mar;144(2-3):243-251.

doi: 10.1007/s00439-023-02624-3. Epub 2024 Jan 3.

Authors

Affiliations

¹ Human Development and Health, Faculty of Medicine, University of Southampton, Southampton, UK.
² Oxford Centre for Genomic Medicine, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
³ Wessex Regional Genetics Laboratory, Salisbury District Hospital, Salisbury, UK.
⁴ Center for Molecular Medicine, Children's Hospital of Fudan University, National Children's Medical Center, Shanghai, 201102, China.
⁵ University of California, Berkeley, Berkeley, CA, 94720, USA.
⁶ Department of Biochemistry and Microbiology, Rutgers University, New Brunswick, NJ, 08873, USA.
⁷ The Jackson Laboratory for Genomic Medicine, 10 Discovery Drive, Farmington, CT, 06032, USA.
⁸ Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX, 77030, USA.
⁹ Agnes Ginges Centre for Molecular Cardiology at Centenary Institute, University of Sydney, Sydney, Australia.
¹⁰ Faculty of Medicine and Health, University of Sydney, Sydney, Australia.
¹¹ Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, MD, USA.
¹² Graduate Program in Biological Sciences and Department of Cell Biology and Molecular Genetics, University of Maryland, College Park, MD, USA.
¹³ Inflammatory Disease Section, National Human Genome Research Institute, Bethesda, MD, USA.
¹⁴ DNAnexus, Mountain View, CA, 94040, USA.
¹⁵ Department of Clinical Genetics, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark.
¹⁶ Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
¹⁷ Laboratoire de Biologie et Génétique du Cancer, Centre François Baclesse, Caen, France.
¹⁸ Inserm U1245, Cancer Brain and Genomics, Normandie Université, UNICAEN, FHU G4 génomique, Rouen, France.
¹⁹ Inserm U1245, Cancer Brain and Genomics, Normandie Université, UNIROUEN, FHU G4 génomique, Rouen, France.
²⁰ Department of Genetics, Univ Rouen Normandie, INSERM U1245, FHU-G4 Génomique and CHU Rouen, 76000, Rouen, France.
²¹ Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02115, USA.
²² Human Development and Health, Faculty of Medicine, University of Southampton, Southampton, UK. d.baralle@soton.ac.uk.
²³ Wessex Clinical Genetics Service, University Hospital Southampton NHS Foundation Trust, Southampton, UK. d.baralle@soton.ac.uk.

PMID: 38170232
PMCID: PMC11976748
DOI: 10.1007/s00439-023-02624-3

Abstract

Variants which disrupt splicing are a frequent cause of rare disease that have been under-ascertained clinically. Accurate and efficient methods to predict a variant's impact on splicing are needed to interpret the growing number of variants of unknown significance (VUS) identified by exome and genome sequencing. Here, we present the results of the CAGI6 Splicing VUS challenge, which invited predictions of the splicing impact of 56 variants ascertained clinically and functionally validated to determine splicing impact. The performance of 12 prediction methods, along with SpliceAI and CADD, was compared on the 56 functionally validated variants. The maximum accuracy achieved was 82% from two different approaches, one weighting SpliceAI scores by minor allele frequency, and one applying the recently published Splicing Prediction Pipeline (SPiP). SPiP performed optimally in terms of sensitivity, while an ensemble method combining multiple prediction tools and information from databases exceeded all others for specificity. Several challenge methods equalled or exceeded the performance of SpliceAI, with ultimate choice of prediction method likely to depend on experimental or clinical aims. One quarter of the variants were incorrectly predicted by at least 50% of the methods, highlighting the need for further improvements to splicing prediction methods for successful clinical application.

PubMed Disclaimer

Conflict of interest statement

Declarations. Competing interests: The authors have no relevant financial or non-financial interests to disclose. On behalf of all authors, the corresponding author states that there is no conflict of interest. Ethics approval: Informed consent was provided for all patients for splicing studies to be conducted. Patients were recruited from Wessex Regional Genetics Laboratory in Salisbury (52 variants) or the Splicing and Disease research study (12 variants) at the University of Southampton, ethically approved by the Health Research Authority (IRAS Project ID 49685, REC 11/SC/0269) and by the University of Southampton (ERGO ID 23056).

Figures

**Fig. 1**
Schematic diagram showing locations of the 56 challenge variants in relation to their nearest splice site, with colour indicating whether (yellow) or not (green) each variant was determined experimentally to impact splicing

**Fig. 2**
Receiver operating characteristic (ROC) curves of model performance based on prediction scores. For Area Under Curve (AUC), see Table 2

**Fig. 3**
Proportion of variants correctly predicted by each method in the different regions (near acceptor, near donor, exonic and intronic distant)

**Fig. 4**
Variants across the splicing region coloured by the number of prediction methods (out of the 12 challenge entrants) that correctly predicted the splicing outcome

See this image and copyright information in PMC

References

1. Cheng J et al (2019) MMSplice: modular modeling improves the predictions of genetic variant effects on splicing. Genome Biol 20(1):48 - PMC - PubMed
1. Danis D, Jacobsen JOB, Carmody LC, Gargano MA, McMurry JA, Hegde A, Haendel MA, Valentini G, Smedley D, Robinson PN (2021) Interpretable prioritization of splice variants in diagnostic next-generation sequencing. Am J Hum Genet 108(9):1564–1577 - PMC - PubMed
1. Ha C, Kim JW, Jang JH (2021) Performance evaluation of SpliceAI for the prediction of splicing of NF1 variants. Genes (basel) 12:1308 - PMC - PubMed
1. Jagadeesh KA et al (2019) S-CAP extends pathogenicity prediction to genetic variants that affect RNA splicing. Nat Genet 51(4):755–763 - PubMed
1. Jaganathan K, Kyriazopoulou Panagiotopoulou S, McRae JF, Darbandi SF, Knowles D, Li YI, Kosmicki JA, Arbelaez J, Cui W, Schwartz GB et al (2019) Predicting splicing from primary sequence with deep learning. Cell 176(3):535–548 - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Predicting the impact of rare variants on RNA splicing in CAGI6

Affiliations

Predicting the impact of rare variants on RNA splicing in CAGI6

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources