Evaluating methods for the prediction of cell-type-specific enhancers in the mammalian cortex

Abstract

Identifying cell-type-specific enhancers is critical for developing genetic tools to study the mammalian brain. We organized the "Brain Initiative Cell Census Network (BICCN) Challenge: Predicting Functional Cell Type-Specific Enhancers from Cross-Species Multi-Omics" to evaluate machine learning and feature-based methods for nominating enhancer sequences targeting mouse cortical cell types. Methods were assessed using in vivo data from hundreds of adeno-associated virus (AAV)-packaged, retro-orbitally delivered enhancers. Open chromatin was the strongest predictor of functional enhancers, while sequence models improved prediction of non-functional enhancers and identified cell-type-specific transcription factor codes to inform in silico enhancer design. This challenge establishes a benchmark for enhancer prioritization and highlights computational and molecular features critical for identifying functional cortical enhancers, advancing efforts to map and manipulate gene regulation in the mammalian cortex.

Keywords: ATAC-seq; DNA sequence model; TF codes; cortex; cross-species; enhancer-AAV; prediction benchmark; single-cell multiomics.

Graphical abstract

Figure 1

Overview of the enhancer prioritization challenge (A) Single-nucleus multi-omics data from the M1 of human, macaque, marmoset, and mouse. mya, million years ago. (B) Schematic of the computational challenge to prioritize candidate cell-type-specific enhancers. (C) Overview of AAV construction, cell-type ATAC-seq specificity, and screening of in vivo activity in the mouse brain for three candidate L5 extratelencephalic-projecting ( ET) enhancers. (D) Teams predicted and ranked 10,000 candidate enhancers for each of 19 cortical cell types and were scored based on prioritization of strong, on-target enhancers. (E) Combinations of data and methods for top team submissions. (F) Normalized benchmark metrics (STAR Methods) based on epifluorescence and SSv4 from in vivo screening.

Figure 2

Comparison of team enhancer rankings (A) Average proportion of ranked enhancers that overlap between pairs of team submissions for all cell types. (B) Upset plot showing the number of validated enhancers that were identified by sets of submissions. (C and D) Rates of identification of (C) on-target and (D) mixed-target, off-target and no-labeling enhancers. (E) Comparison of methods based on distributions of normalized enrichment scores (NES). For each method and cell-type-specific ranking, NES measures the area under the recovery curve (AUC) up to the 1,000^th element compared to a random ranking. (F) Heatmap ordered by Aerts scATAC_triplet scoring of L5 ET enhancers and summary of validation results. Examples of a strong (AiE0456m) and weak (AiE0460m) enhancer with Pou3f1 motifs identified by the CREsted model in the highlighted region. AiE0456m was also validated with SSv4.

Figure 3

Enhancer features predictive of functional activity (A) Comparison of molecular features between on target and other enhancer categories. ∗∗p < 0.001, ∗∗∗p < 0.0001 Wilcoxon rank-sum test, two sided, unpaired. (B) Correlation of H3K27ac and ATAC-seq specificity for astrocyte enhancers. (C) Examples of astrocyte enhancers with in vivo activity that is better predicted by H3K27ac than ATAC-seq signal. (D) Summary of informative features from a random forest model predicting enhancer activity. ANOVA with Tukey post hoc tests, Bonferroni-corrected p values. (E) Schematic of ATAC-seq peak quantification based on cut sites or coverage and boxplot comparison of peak specificity for all on-target enhancers between different preprocessing methods. Adjusted p values were obtained through t tests, ∗p < 0.05, ∗∗p < 0.01. (F) Overall enhancer activity prediction performance from peak specificity for the different methods for on-target (left) and all except off-target (right) enhancers.

Figure 4

Refinement of models and enhancer screening results (A) t-distributed stochastic neighbor embedding (t-SNE) plots of enhancers based on ATAC-seq specificity and labeled by the targeted cell type. On-target and no-labeling enhancers had explainable or unexplainable cell type labeling patterns based on ATAC-seq and DNA sequence (CREsted) model predictions. (B) River plots of enhancer activity, predictions, and rescoring of experimental validation data. (C and D) Model scores, predicted TF motifs, and SYFP fluorescence for two oligo enhancers with epifluorescence strengths (C) strong on-target and (D) no-labeling rescored to weak on-target activity. (E) Performance of enhancer ranking methods using the rescored enhancer activities. AP, average precision. scATAC included two normalizations: count-normalized coverage pseudobulk or peak scaled. (F) CREsted model scores for strong and weak on-target enhancers grouped by cell type. Mean ± SEM. (G) Comparison of models at identifying No-Labeling enhancers. ∗p < 0.05, ∗∗∗p < 0.001, Wilcoxon rank-sum test, Bonferroni-corrected p values.