NEURD offers automated proofreading and feature extraction for connectomics

Abstract

We are in the era of millimetre-scale electron microscopy volumes collected at nanometre resolution^1,2. Dense reconstruction of cellular compartments in these electron microscopy volumes has been enabled by recent advances in machine learning^3-6. Automated segmentation methods produce exceptionally accurate reconstructions of cells, but post hoc proofreading is still required to generate large connectomes that are free of merge and split errors. The elaborate 3D meshes of neurons in these volumes contain detailed morphological information at multiple scales, from the diameter, shape and branching patterns of axons and dendrites, down to the fine-scale structure of dendritic spines. However, extracting these features can require substantial effort to piece together existing tools into custom workflows. Here, building on existing open source software for mesh manipulation, we present Neural Decomposition (NEURD), a software package that decomposes meshed neurons into compact and extensively annotated graph representations. With these feature-rich graphs, we automate a variety of tasks such as state-of-the-art automated proofreading of merge errors, cell classification, spine detection, axonal-dendritic proximities and other annotations. These features enable many downstream analyses of neural morphology and connectivity, making these massive and complex datasets more accessible to neuroscience researchers.

Fig. 1. Working with neuronal meshes from large-volume electron microscopy segmentations.

a, The MICrONS Minnie65 volume is an approximately 1,300 × 820 × 520 μm³ rectangular volume from mouse visual cortex; H01 is a wedge-shaped volume from human temporal cortex with a longest dimension of 3 mm, a width of 2 mm and a thickness of 150 μm. b–e, The range of accuracy across neural reconstructions in the MICrONS and H01 volumes. b, Example of a nearly complete (manually proofread) single neuron. c, A mesh containing two merged neurons from the MICrONS volume. d, Example of an orphan merge error with a piece of dendrite incorrectly merged onto a neuron mesh. e, An incompletely reconstructed neuron from the H01 volume. f, An overview of the NEURD workflow: starting from volumes and their initial mesh states (Fig. 1), to the processing pipeline (Fig. 2), automatic proofreading (Fig. 3), and cell typing (Supplementary Fig. 12), which then enables the analysis of morphology (Fig. 4), connectomics, and functional connectomics (Fig. 5).

Fig. 2. Decomposition, feature extraction and graph annotation.

a, The input data (mesh and synapses) required for the NEURD workflow. b, The reconstructed meshes are pre-processed through a number of steps including decimation, glia and nuclei removal, soma detection and skeletonization. Mesh features are projected back onto the skeleton and spines are detected. c, Decomposition graph object composed of two neurons merged together. The decomposition compresses the skeleton, mesh and synapse annotations of a non-branching segment into a single node in a graph, with directed edges to the downstream segments connected at a branch point. The soma is the singular root node of this tree. d, NEURD automates computation of features at multiple levels. Node (non-branching segment)-level features include basic mesh characteristics (for example, diameter of the neural process or number of synapses per skeletal length). Subgraph features capture relationships between adjacent nodes such as branching angle or width differences. Graph features capture characteristics of the entire neuron and are computed by weighted average or sum of node features, or by counting subgraph motifs. Postsyn, postsyaptic region. e, The final product is a cleaned and annotated decomposition object with a single soma that can be fed into a variety of downstream analyses. f, NEURD supports a variety of operations and manipulations on the decomposition objects. Multi-soma splitting is performed with heuristic rules. The entire decomposition graph is classified as excitatory or inhibitory and one subgraph is identified as the axon. Automated proofreading is performed to remove probable merge errors (see Fig. 3). A set of heuristic rules is implemented to label neural compartments, followed by a finer-scale cell-type classification using graph neural networks (GNNs) (Supplementary Fig. 12). PCA, principal components analysis.

Fig. 3. NEURD graph decomposition enables automated proofreading.

a, Implementing domain knowledge as subgraph rules to automatically remove merge errors (see Supplementary Fig. 3 for more rules). b, Laminar distribution of merge errors (H01). The inhomogeneity of errors across different layers, possibly due to differences in neuropil density. The pial surface is to the right and slightly up (see Supplementary Fig. 7 for more details). c, Increased frequency of axon edits is observed in layer 5 of cortex (MICrONS). Pial surface is up. d, Dendritic errors in the MICrONS dataset increase near the top layers of the volume, where fine excitatory apical tufts lead to more frequent merges (see Supplementary Fig. 6 for more details). e,f, MICrONS (e) and H01 (f) synapse validation quantified by synapse precision and recall compared with manual proofreading (ground truth). ‘Before’ describes the accuracy of the raw segmentation prior to any proofreading. The substantial increases in precision ‘After’ automated proofreading (especially for axons) indicates that the cleaned neurons have good fidelity. The reduction in ‘After’ recall indicates the loss of some valid synapses in the automatic proofreading process (mostly concerning axons), while still retaining the majority of correct synapses (see also Supplementary Fig. 9 restricted to single somas). Dend, dendrite. g, An excitatory neuron from the MICrONS dataset in the 75th percentile of merge error skeletal length; identified merge errors are shown in red. h–k, Number of true-positive (TP) and false-positive (FP) axonal synapses from individual excitatory (h,i) or inhibitory (j,k) neurons in the validation set before (h,j) and after (i,k) automated proofreading, illustrating the large number of false-positive (red) synapses in the raw segmentation that are removed by automated proofreading (see Supplementary Figs. 8 and  9 for more details on the MICrONS dataset and Supplementary Fig. 10 for similar validation on the H01 dataset).

Fig. 4. Morphological analysis enabled by NEURD feature extraction.

a, Average number of synapses onto the AIS of cells at different laminar depths (mean ± s.d.) for MICrONS (n = 22,955). b, Distribution of the number of soma synapses per cell. As expected, neurons in the MICrONS volume have more identified synapses onto their soma, despite the smaller surface area compared to human somas (see Supplementary Fig. 16 for more AIS and soma synapse results). c, An example spine segmentation with the features extracted for each spine submesh followed by a kernel density estimation of the UMAP embedding of these features for spines sampled from the MICrONS dataset (see Supplementary Fig. 19 for more details). Exc, excitatory; inh, inhibitory. d, Histograms showing the distribution of the mean skeletal angle of the thickest basal stem as a function of volume depth (see Supplementary Fig. 17 for more details). e, Spine head synapse size and spine head volume joint distribution for the H01 dataset, showing a positive Pearson’s correlation coefficient (P < 10⁻³⁰⁰; see Supplementary Fig. 20 for more details). f, Histogram for all the non-apical dendritic stems of every neuron in the MICrONS volume comparing the initial width of the stem to the number of leaf nodes (blue scaling indicates the number of dendritic stems for a given bin), showing a positive Pearson’s correlation coefficient (corr; P < 10⁻³⁰⁰; see Supplementary Fig. 18 for more details). g, The ratio of non-spine synapses to spine counts varies across cell types. h, Distribution of spine characteristics for different cell types, comparing the volume of each spine head and the size of the largest synapse on that spine head, where outlines indicate the quartile boundaries for each distribution (see Supplementary Fig. 15 for more cell-type distributions). See Supplementary Table 2 for all associated n values.

Fig. 5. Connectivity analysis enabled by NEURD.

a, Schematic illustrating two proximities between a pair of neurons (axon passing within 5 μm radius of a dendrite). Only one proximity has a synapse, thus the ‘conversion rate’ is 50%. b, Cumulative density function (CDF) of the postsynaptic dendritic skeletal walks for different connection types, demonstrating that excitatory inputs occur further along the dendrite from the soma (see Supplementary Fig. 23 for more details). c, Mean conversion rate as a function of distance along the axon (see Supplementary Fig. 23 for more details). d,e, Conversion rates (synapses/proximities) for different excitatory and inhibitory combinations. The x-axis represents the maximum distance that is considered a proximity. f, Conversion rates for different cell-type subclasses and compartments are largely consistent with previous studies (MICrONS; cell-type labels generated from a GNN classifier; see Supplementary Table 1 for glossary and Supplementary Fig. 24 for more conversion rates). g, The frequency (mean ± s.d.) of reciprocal connections or edge-dense three-node motifs was enriched compared with null distributions where synaptic degree distribution is held the same but edges are shuffled (orange), where the synaptic edges are shuffled across existing proximity edges (green) or where synapses are randomly shuffled (red); 250 random graph samples for each null distribution comparison (see Supplementary Fig. 25 for more details and inhibitory/excitatory-only graphs). h, Example multi-synaptic connection (n = 7 synapses) from an excitatory to inhibitory neuron (H01). i, Distribution of response correlation mean (±s.e.m.) between pairs of functionally matched excitatory neurons (MICrONS). Response correlation is significantly larger for pairs of neurons with 4 or more synapses connecting them (n = 11 pairs) compared with those with 1, 2 or 3 synapses (n = 5,350, 280 or 34 pairs, respectively; see Supplementary Fig. 27 for more details). See Supplementary Table 2 for all associated n values.