A hybrid YOLO-UNet3D framework for automated protein particle annotation in Cryo-ET images

Abstract

Accurate localization and identification of protein complexes in cryo-electron tomography (cryo-ET) volumes are essential for understanding cellular functions and disease mechanisms. However, automated annotation of these macromolecular assemblies remains challenging due to low signal-to-noise ratios, missing wedge artifacts, heterogeneous backgrounds, and structural diversity. In this study, we present a hybrid framework integrating You Only Look Once (YOLO) object detection with UNet3D volumetric segmentation, enhanced by density-based spatial clustering of applications with noise (DBSCAN) post-processing for automated protein particle annotation in cryo-ET volumes. Our approach combines YOLO's efficient region proposal capabilities with UNet3D's powerful 3D feature extraction through a dual-branch architecture featuring optimized Spatial Pyramid Pooling-Fast (SPPF) modules and asymmetric feature splitting. Extensive experiments on the Chan Zuckerberg Initiative Imaging (CZII) cryo-ET dataset demonstrate that our method significantly outperforms existing state-of-the-art approaches, including DeepFinder, standard UNet3D, YOLOv5-3D, and 3D ResNet models, achieving a mean recall of 0.8848 and F4-score of 0.7969. The framework demonstrates robust performance across various protein particle types and imaging conditions, offering a promising technical solution for high-throughput structural biology workflows requiring accurate macromolecular annotation in cellular cryo-ET data.

Keywords: Cryo-electron tomography; Deep learning; Density clustering; Protein particle detection; UNet3D; YOLO.

Fig. 1

Overview of the 3D Cryo-ET particle detection system. The figure shows: (Left) The complete processing pipeline for 3D Cryo-ET tomogram data, including 2D slice extraction (640640), resizing operations, and YOLO-based detection with visualization of particle identification results; (a) The modified YOLO detection network architecture featuring SPPF (Spatial Pyramid Pooling Fast) modules, C2f blocks, convolution layers, and upsampling operations, culminating in the detection module for 3D coordinate and particle classification output; (b) The feature extraction and processing network showing multi-scale feature maps (64, 128, 256, 512 channels) with RCSP (Residual CSP) modules, CBM4 blocks, Conv2D layers, and FPA (Feature Pyramid Aggregation) for comprehensive particle feature analysis.

Fig. 2

Key Building Blocks of the Proposed Network. (a) Residual Unit (ResUnit): Architecture of the residual unit used in the UNet3D branch. (b) Asymmetric Split-and-Fuse Block (C2f Module):Structure of the asymmetric C2f module with a 1:2 feature split and dense convolutional path. (c) Fast Spatial Pyramid Pooling (SPPF) Module: Layout of the multi-branch SPPF module with adaptive max pooling for multi-scale feature fusion.

Fig. 3

Central slab from a representative tomogram of the in vitro sample with selected set of annotations. Different colors indicate different particle types.

Fig. 4

Visualization of dataset structure. (a) shows tomographic image slices; (b) and (c) display 3D scatter plots of protein particle annotations in two tomograms. These visualizations highlight the volumetric complexity and distribution diversity present in the dataset.

Fig. 5

Visualization of feature maps at different stages of the model. The enhanced SPPF module enables effective multi-scale feature extraction, particularly beneficial for particles of varying sizes.

Fig. 6

Analysis of model performance under different noise levels and imaging conditions. The model maintains robust performance even under challenging conditions, demonstrating its practical applicability.