CONFIRMS: A Toolkit for Scalable, Black Box Connectome Assessment and Investigation

Abstract

The nanoscale connectomics community has recently generated automated and semi-automated "wiring diagrams" of brain subregions from terabytes and petabytes of dense 3D neuroimagery. This process involves many challenging and imperfect technical steps, including dense 3D image segmentation, anisotropic nonrigid image alignment and coregistration, and pixel classification of each neuron and their individual synaptic connections. As data volumes continue to grow in size, and connectome generation becomes increasingly commonplace, it is important that the scientific community is able to rapidly assess the quality and accuracy of a connectome product to promote dataset analysis and reuse. In this work, we share our scalable toolkit for assessing the quality of a connectome reconstruction via targeted inquiry and large-scale graph analysis, and to provide insights into how such connectome proofreading processes may be improved and optimized in the future. We illustrate the applications and ecosystem on a recent reference dataset.Clinical relevance- Large-scale electron microscopy (EM) data offers a novel opportunity to characterize etiologies and neurological diseases and conditions at an unprecedented scale. EM is useful for low-level analyses such as biopsies; this increased scale offers new possibilities for research into areas such as neural networks if certain bottlenecks and problems are overcome.

Fig. 1.

Synthetic examples illustrating common segmentation errors resulting from automated segmentation methods [22], [4] (a: ground truth, b: false merge, c: false split)

Fig. 2.

A synapse task in progress with the keypoint “node” annotation web application. In this example, the task instructs the user to annotate the center of each synapse in the 3D electron microscopy volume. A running count and index of synapse annotations are accessible to the user.

Fig. 3.

A forced-choice synapse proofreading task in progress. In this example, the task loads the image volume at the center of the annotated synapse of interest. The user can interact with the “yes,” “no,” and “maybe” buttons, or use keyboard shortcuts to complete the verification task.

Fig. 4.

A tracing task in progress with the Skeleton “graph” annotation web application. In this example, the task instructs the user to trace the postsynaptic side of the marked synapse. The user can place intermediate “breadcrumb” nodes while navigating the 3D volume to maintain the structure of the neuron while annotating keypoints at each synaptic connection. A running count and locations of postsynaptic synapses annotated is accessible to the user.

Fig. 5.

A forced-choice Skeleton “graph” proofreading task in progress. In this example, the task loads the image volume at the center of an annotated graph of interest. The user can interact with the “yes”, “no”, “maybe” buttons or use keyboard shortcuts to complete the verification task.

Fig. 6.

Fused novice Skeleton “graph” annotations with 3D meshed segmentation, illustrating the product of our visualizations with various overlays and also some of our visualization tools.

Fig. 7.

Overall CONFIRMS ecosystem connecting our proofreading and analysis tools into an evaluation workflow to assess connectome quality for downstream inference, based on an unseen (or “black box”) processing pipeline.