Evaluating the informativeness of deep learning annotations for human complex diseases

Abstract

Deep learning models have shown great promise in predicting regulatory effects from DNA sequence, but their informativeness for human complex diseases is not fully understood. Here, we evaluate genome-wide SNP annotations from two previous deep learning models, DeepSEA and Basenji, by applying stratified LD score regression to 41 diseases and traits (average N = 320K), conditioning on a broad set of coding, conserved and regulatory annotations. We aggregated annotations across all (respectively blood or brain) tissues/cell-types in meta-analyses across all (respectively 11 blood or 8 brain) traits. The annotations were highly enriched for disease heritability, but produced only limited conditionally significant results: non-tissue-specific and brain-specific Basenji-H3K4me3 for all traits and brain traits respectively. We conclude that deep learning models have yet to achieve their full potential to provide considerable unique information for complex disease, and that their conditional informativeness for disease cannot be inferred from their accuracy in predicting regulatory annotations.

Fig. 1. Summary of disease informativeness of allelic-effect deep learning annotations.

We report the number of allelic-effect annotations with significant heritability enrichment, marginal conditional τ^⋆, and joint conditional τ^⋆, across a different deep learning models (DeepSEA/Basenji), b different aggregation strategies (Avg/Max) and c different chromatin marks (DNase/H3K27ac/H3K4me1/H3K4me3). Numerical results are reported in Supplementary Table 5 (numerical summary of results), Supplementary Table 6 (enrichment and marginal τ^⋆ for all tissues, all traits analysis), Supplementary Table 15 (enrichment and marginal τ^⋆ of blood cell types, blood traits analysis), Supplementary Table 21 (enrichment and marginal τ^⋆ of brain tissues, brain traits analysis) and Supplementary Table 27 (joint τ^⋆ of brain tissues, brain traits analysis). No Supplementary Table is needed for joint τ^⋆ of all tissues, all traits (1 marginally significant annotation) or blood cell types, blood traits (0 marginally significant annotations).

Fig. 2. Disease informativeness of non-tissue-specific allelic-effect deep learning annotations.

a Heritability enrichment, conditioned on the non-tissue-specific variant-level joint model. Horizontal line denotes no enrichment. b Standardized effect size τ^⋆ conditioned on either the non-tissue-specific variant-level joint model (marginal analysis: left column, white) or the variant-level joint model plus 1 non-tissue-specific allelic-effect Basenji annotation (BasenjiΔ-H3K4me3-Max) (non-tissue-specific final joint model: right column, dark shading. Results are meta-analyzed across 41 traits. Results are displayed only for the allelic-effect annotation (BasenjiΔ-H3K4me3-Max) with significant τ^⋆ in marginal analyses after correcting for 106 (variant-level + allelic-effect) non-tissue-specific annotations tested (P < 0.05/106), along with the corresponding variant-level annotation; the correlation between the two annotations is 0.43. For non-tissue-specific final joint model (right column), **P < 0.05/106. Error bars denote 95% confidence intervals. Numerical results are reported in Supplementary Table 6 and Supplementary Table 8.

Fig. 3. Disease informativeness of brain-specific allelic-effect deep learning annotations.

a Heritability enrichment, conditioned on the brain-specific variant-level joint model and the 1 significant non-tissue-specific allelic-effect annotation (BasenjiΔ-H3K4me3-Max). Horizontal line denotes no enrichment. b Standardized effect size τ^⋆ conditioned on either the brain-specific variant-level joint model and BasenjiΔ-H3K4me3-Max (marginal analysis: left column, white) or the same model plus 1 brain-specific allelic-effect annotation (BasenjiΔ-H3K4me3-brain-Max) (brain-specific final joint model: right column, dark shading). Results are meta-analyzed across 8 brain-related traits. Results are displayed only for the 2 allelic-effect annotations with significant τ^* in marginal analyses after correcting for 80 (variant-level + allelic-effect) brain-specific annotations tested (P < 0.05/80), along with the corresponding variant-level annotations; the correlation between the two allelic-effect annotations is 0.78, and the average correlation between the two pairs of variant-level (Basenji) and allelic-effect (BasenjiΔ) annotations is 0.44. For brain-specific final joint model (right column), **P < 0.05/80. Error bars denote 95% confidence intervals. Numerical results are reported in Supplementary Table 21 and Supplementary Table 27.