INDELpred: Improving the prediction and interpretation of indel pathogenicity within the clinical genome

Abstract

Small insertions and deletions (indels) are critical yet challenging genetic variations with significant clinical implications. However, the identification of pathogenic indels from neutral variants in clinical contexts remains an understudied problem. Here, we developed INDELpred, a machine-learning-based predictive model for discerning pathogenic from benign indels. INDELpred was established based on key features, including allele frequency, indel length, function-based features, and gene-based features. A set of comprehensive evaluation analyses demonstrated that INDELpred exhibited superior performance over competing methods in terms of computational efficiency and prediction accuracy. Importantly, INDELpred highlighted the crucial role of function-based features in identifying pathogenic indels, with a clear interpretability of the features in understanding the disease-causing variants. We envisage INDELpred as a desirable tool for the detection of pathogenic indels within large-scale genomic datasets, thereby enhancing the precision of genetic diagnoses in clinical settings.

Keywords: InDel; clinical genomics; machine learning; pathogenicity prediction; whole genome sequencing.

Figure 1

Overview of INDELpred development and evaluation (A) After annotation by ANNOVAR, the function-based features for indel variants were represented by the studentized residuals that were derived from the fitted ordinary least squares regression models, while the gene-based features were calculated by the ratios of pathogenic to all indels for each function in the specific gene regions. (B) With the four distinct feature categories, the GBDT-based INDELpred model was trained upon the ClinVarTrain dataset using the stratified 5-fold cross-validation. The final INDELpred model was subsequently applied to four testing datasets for the evaluation of prediction accuracy, runtime, and data storage requirements using a set of metrics. (C) INDELpred model for indel pathogenicity prediction was further validated across various genomic contexts, including stability analysis with different genomic factors, reliability analysis based on the varying support laboratories (confidence levels) of indels, shared gene analysis between HGMD and gnomAD, and the adoption of a clinical WGS dataset with 30 pediatric individuals for practical applicability assessment.

Figure 2

Indel pathogenicity prediction performance comparison between INDELpred and the competing methods on the datasets of ClinVarTest, VKGLTest, and HGMDTest (A) The number of predicted loci by different models for the three testing datasets. The gray color indicates the loci not predicted by the model. (B) Radar chart of six evolution metrics. Values closer to the periphery represent a score approaching 1, indicating the high prediction performance. (C) ROC curves with AUROC values yielded by different methods. (D) Distribution of prediction scores by different models. The horizontal axis ranges from 0 to 1. CADD scores were normalized as ${\text{score}}_{\text{CADD}}\times 0.5/20$, where we chose the value of 20 as the CADD cutoff value. (E) Time consumption of the three models across the three datasets. Each method was independently executed five times on each dataset to ensure the consistency of the results. The data storage requirement for each method is shown on the x axis. Statistical test: two-sided Mann-Whitney U test with the Benjamini–Hochberg correction. ∗∗p < 0.01 and ∗∗∗∗p < 0.0001. The center line indicating the median and whiskers representing 1.5 × IQR.

Figure 3

Evaluation of INDELpred for the impact of different genomic factors (A–E) AUROC results assessed for each partition subset of the ClinVarTest dataset based on genomic factors of whether or not the indel length was divisible by three (A), indel length (B), AF (C), groups of TSGs and ONCs (D), and groups of shared genes (E). The number of indels in each subset is indicated on the top accordingly. (F) Radar chart of the six metrics valued using the HGMDSharedGene dataset. (G) AUROC results with respect to different levels of CLINSTAT credibility in ClinVarTest (left) as well as different support laboratories in VKGLTest (right).

Figure 4

Comparison of indel pathogenicity prediction performance between INDELpred and the competing methods on the clinical WGS dataset (A) Computational time taken by the three methods for the clinical WGS dataset collected from 30 pediatric individuals. Statistical test: two-sided Mann-Whitney U test with the Benjamini-Hochberg correction. ∗∗∗∗p < 0.0001. Each box in the boxplots corresponds to the interval between the 25th and 75th percentile (interquartile range, IQR) with the center line indicating the median and whiskers representing 1.5 × IQR. (B) Precision-recall curves with AUPR values for each predictive model. (C) Curve plot showing sensitivity of pathogenic indel in the top-k variants ranked by different prediction methods for varying k values. (D) Bar plot showing sensitivity of individual in top 30, top 100, and top 150 variants ranked by different prediction methods. (E) Prediction scores yielded by different predictive models for the 26 disease-causative indels. CADD scores were normalized as ${\text{score}}_{\text{CADD}}\times 0.5/20$, where we chose the value of 20 as the CADD cutoff value. Each box in the boxplots corresponds to the interval between the 25th and 75th percentile (interquartile range, IQR) with the center line indicating the median and whiskers representing 1.5 × IQR.

Figure 5

Analysis of pathogenicity-related factors through the lens of features utilized by INDELpred (A) Spearman correlation coefficient matrix of the 16 features used by INDELpred. (B) The contribution of each individual feature, represented by feature importance, toward the prediction of indel pathogenicity by INDELpred model. (C) Relative contribution of each of four feature categories toward the prediction of indel pathogenicity by INDELpred model. Each box in the boxplots corresponds to the interval between the 25th and 75th percentile (interquartile range, IQR) with the center line indicating the median and whiskers representing 1.5 × IQR. (D) Comparison of AUROC results across three datasets for AF with threshold-free and threshold-based values. (E) Hierarchical clustering analysis of 16 individual features. All values of individual features were Z scored.