Explicable prioritization of genetic variants by integration of rule-based and machine learning algorithms for diagnosis of rare Mendelian disorders

Abstract

Background: In the process of finding the causative variant of rare diseases, accurate assessment and prioritization of genetic variants is essential. Previous variant prioritization tools mainly depend on the in-silico prediction of the pathogenicity of variants, which results in low sensitivity and difficulty in interpreting the prioritization result. In this study, we propose an explainable algorithm for variant prioritization, named 3ASC, with higher sensitivity and ability to annotate evidence used for prioritization. 3ASC annotates each variant with the 28 criteria defined by the ACMG/AMP genome interpretation guidelines and features related to the clinical interpretation of the variants. The system can explain the result based on annotated evidence and feature contributions.

Results: We trained various machine learning algorithms using in-house patient data. The performance of variant ranking was assessed using the recall rate of identifying causative variants in the top-ranked variants. The best practice model was a random forest classifier that showed top 1 recall of 85.6% and top 3 recall of 94.4%. The 3ASC annotates the ACMG/AMP criteria for each genetic variant of a patient so that clinical geneticists can interpret the result as in the CAGI6 SickKids challenge. In the challenge, 3ASC identified causal genes for 10 out of 14 patient cases, with evidence of decreased gene expression for 6 cases. Among them, two genes (HDAC8 and CASK) had decreased gene expression profiles confirmed by transcriptome data.

Conclusions: 3ASC can prioritize genetic variants with higher sensitivity compared to previous methods by integrating various features related to clinical interpretation, including features related to false positive risk such as quality control and disease inheritance pattern. The system allows interpretation of each variant based on the ACMG/AMP criteria and feature contribution assessed using explainable AI techniques.

Keywords: Clinical genome interpretation; Explainable AI; Mendelian disorder; Variant prioritization.

Fig. 1

Overview of 3ASC variant prioritization system. The Bayesian score, symptom similarity, the 3Cnet score, and false risk features were trained to build different 3ASC models including the baseline model, logistic regression models, and random forest models. 3ASC prioritizes variants of each patient according to the scores, and annotates ACMG criteria to each variant. The user can interpret the prioritization result based on the ACMG rules and feature contribution for each variant

Fig. 2

Performance comparison between 3ASC models using cross-validation. A Average ROC curve. True positive rates from different folds were averaged for each false positive rate using interpolation. False positive rates ranging from 0 to 0.1 were plotted because the true positive rates were mostly saturated afterwards. B Average PR curve. C Average top-k recall

Fig. 3

Top-k recall comparison with benchmark models using external validation. The same set of genes and variants were used to compare the performance of variant prioritization without any bias. For Exomiser and LIRICAL, gene scores were first used to prioritize the most probable causal genes and then variants were prioritized using variant scores

Fig. 4

Feature importance measured using SHAP and MDA. A SHAP force plot of 3ASC_RF_ALL; B Feature importance of 3ASC_RF_ALL

Fig. 5

SHAP plot of Individual variants in a patient with hemophilia A. A Force plot of a false call variant; B Force plot of a confirmative variant