Evaluation of phenotype-driven gene prioritization methods for Mendelian diseases

Abstract

It's challenging work to identify disease-causing genes from the next-generation sequencing (NGS) data of patients with Mendelian disorders. To improve this situation, researchers have developed many phenotype-driven gene prioritization methods using a patient's genotype and phenotype information, or phenotype information only as input to rank the candidate's pathogenic genes. Evaluations of these ranking methods provide practitioners with convenience for choosing an appropriate tool for their workflows, but retrospective benchmarks are underpowered to provide statistically significant results in their attempt to differentiate. In this research, the performance of ten recognized causal-gene prioritization methods was benchmarked using 305 cases from the Deciphering Developmental Disorders (DDD) project and 209 in-house cases via a relatively unbiased methodology. The evaluation results show that methods using Human Phenotype Ontology (HPO) terms and Variant Call Format (VCF) files as input achieved better overall performance than those using phenotypic data alone. Besides, LIRICAL and AMELIE, two of the best methods in our benchmark experiments, complement each other in cases with the causal genes ranked highly, suggesting a possible integrative approach to further enhance the diagnostic efficiency. Our benchmarking provides valuable reference information to the computer-assisted rapid diagnosis in Mendelian diseases and sheds some light on the potential direction of future improvement on disease-causing gene prioritization methods.

Keywords: HPO; Mendelian diseases; benchmarking; gene prioritization.

Figure 1

Illustration of study workflow. Flowchart of data collection and method implementation in this work. DDD patient cohort includes 305 cases with developmental disorders (represented as light blue) while the in-house KMCGD patient cohort involves a total of 209 cases with a wide range of syndromes (represented as various colors). Then, curated HPO terms and a VCF file of each case in both cohorts are imported into six ‘HPO + VCF’ prioritization methods. Additionally, curated HPO terms of each case are imported into five ‘HPO-only’ prioritization methods. In particular, AMELIE is run in both ‘HPO + VCF’ mode and ‘HPO-only’ mode(AMELIE_HPO). Finally, for each case, the ranking position of the known causal gene in the gene list output by each method is recorded, based on which the performance of each method is evaluated.

Figure 2

Distribution plots of performance evaluation results. Distribution plots of performance evaluation results of 10 phenotype-driven gene prioritization methods on the DDD (A) and KMCGD (B) datasets. The distribution plots illustrate the percentage of the cases with causal genes ranked in top-1 and within the top-5, -10, -20, -30, -40 and -50 by each method. Each method is represented by a different color.

Figure 3

CDF and bar plots of performance evaluation results. CDF plots (A) and bar plots (B) of performance evaluation results of 10 phenotype-driven gene prioritization methods on the DDD (left) and KMCGD (right) datasets. The CDF plots illustrate the percentage of the cases with causal genes ranked within the top k by each method. k could be any integer between 1 and 50 (inclusive). Each method is represented by a different color. The bar plots illustrate the relative proportion of each group involved cases with causal genes ranked within a designated range. Each group is represented by a different color. (C) The overlapping set of cases with causal genes ranked in top-1 and within top-5, -10, -20, -30, -40 and -50 by LIRICAL and AMELIE in DDD (left) and KMCGD (right) dataset.

Figure 4

Performance evaluation across different disease subgroups. (A) Frequency distribution of the HPO parent classes for the KMCGD dataset. The HPO terms of each case in the KMCGD dataset are assigned to HPO parent classes according to the official HPO hierarchy and some cases involve more than one kind of HPO parent class. (B) Disease subgroup composition of KMCGD dataset. Case amount and proportion are tagged for each subgroup. (C) CDF plots of performance evaluation results of 10 phenotype-driven gene prioritization methods on each subgroup of the KMCGD dataset.