Evaluating Methods for the Prediction of Cell Type-Specific Enhancers in the Mammalian Cortex

Abstract

Identifying cell type-specific enhancers in the brain is critical to building genetic tools for investigating the mammalian brain. Computational methods for functional enhancer prediction have been proposed and validated in the fruit fly and not yet the mammalian brain. We organized the 'Brain Initiative Cell Census Network (BICCN) Challenge: Predicting Functional Cell Type-Specific Enhancers from Cross-Species Multi-Omics' to assess machine learning and feature-based methods designed to nominate enhancer DNA sequences to target cell types in the mouse cortex. Methods were evaluated based on in vivo validation data from hundreds of cortical cell type-specific enhancers that were previously packaged into individual AAV vectors and retro-orbitally injected into mice. We find that open chromatin was a key predictor of functional enhancers, and sequence models improved prediction of non-functional enhancers that can be deprioritized as opposed to pursued for in vivo testing. Sequence models also identified cell type-specific transcription factor codes that can guide designs of in silico enhancers. This community challenge establishes a benchmark for enhancer prioritization algorithms and reveals computational approaches and molecular information that are crucial for identifying functional enhancers in mammalian cortical cell types. The results of this challenge bring us closer to understanding the complex gene regulatory landscape of the mammalian cortex and to designing more efficient genetic tools to target cortical cell types.

Keywords: AAV; ATAC-seq; DNA methylation; HiC; RNA-Seq; benchmark; cell types; challenge; cortex; enhancer; machine learning; mouse; multiome; primates.

Figure 1.. Overview of the enhancer prioritization challenge.

(A) Single nucleus multi-omic data from primary motor cortex of human, macaque, marmoset, and mouse. mya, million years ago. (B) Schematic of the computational challenge to prioritize candidate cell type-specific enhancers. (C) Overview of AAV construction, cell type ATAC-seq specificity, and screening of in vivo activity in the mouse brain for three candidate L5 ET enhancers. (D) Teams predicted and ranked 10,000 candidate enhancers for each of 19 cortical cell types and were scored based on prioritization of strong, On-Target enhancers. (E) Combinations of data and methods for top team submissions. (F) Normalized benchmark metrics (Methods) based on epifluorescence and SSv4 from in vivo screening.

Figure 2.. Comparison of team enhancer rankings.

(A) Average proportion of ranked enhancers that overlap between pairs of team submissions for all cell types. (B) Upset plot showing the number of validated enhancers that were identified by sets of submissions. (C,D) Rates of identification of (C) On-Target and (D) Mixed-Target, Off-Target and No-Labeling enhancers. (E) Comparison of methods based on distributions of normalized enrichment scores (NES). For each method and cell type specific ranking, NES measures the area under the recovery curve (AUC) up to the 1,000^th element compared to a random ranking. (F) Heatmap ordered by Aerts scATAC_triplet scoring of L5 ET enhancers and summary of validation results. Examples of a strong (AiE0456m) and weak (AiE0460m) enhancer with Pou3f1 motifs identified by the CREsted model in the highlighted region. AiE0456m was also validated with SSv4.

Figure 3.. Enhancer features predictive of functional activity.

(A) Comparison of molecular features between On-Target and other enhancer categories. ** P < 0.001, *** P < 0.0001 Wilcoxon rank-sum test two-sided, unpaired. (B) Correlation of H3K27ac and ATAC-seq specificity for astrocyte enhancers. (C) Examples of astrocyte enhancers with in vivo activity that is better predicted by H3K27ac than ATAC-seq signal. (D) Summary of informative features from a Random Forest model predicting enhancer activity. ANOVA with Tukey post hoc tests, Bonferroni-corrected P-values. (E) Schematic of ATAC-seq peak quantification based on cut sites or coverage. Box plot comparison of peak specificity for all on-target enhancers between different preprocessing methods. Adjusted p-values were obtained through t-tests, *P < 0.05, **P < 0.01. (F) Overall enhancer activity prediction performance from peak specificity for the different methods for On-target (left) and all, except Off-target, (right) enhancers.

Figure 4.. Refinement of models and enhancer screening results.

(A) tSNE plots of enhancers based on ATAC-seq specificity and labeled by the targeted cell type. On-Target and No-Labeling enhancers had explainable or unexplainable cell type labeling patterns based on ATAC-seq and DNA sequence (CREsted) model predictions. (B) River plots of enhancer activity, predictions and rescoring of experimental validation data. (C-D) Model scores, predicted TF motifs and SYFP fluorescence for two Oligo enhancers with epifluorescence strengths (C) strong On-Target and (D) No-Labeling rescored to weak On-Target activity. (E) Performance of enhancer ranking methods using the rescored enhancer activities. AP, average precision. scATAC included two normalizations: count-normalized coverage pseudobulk or peak-scaled. (F) CREsted model scores for strong and weak On-Target enhancers grouped by cell type. Mean +/− SEM. (G) Comparison of models at identifying No-Labeling enhancers. * P < 0.05, *** P < 0.001, Wilcoxon rank-sum test, Bonferroni-corrected P-values.