AI analysis of super-resolution microscopy: Biological discovery in the absence of ground truth

Abstract

Super-resolution microscopy, or nanoscopy, enables the use of fluorescent-based molecular localization tools to study molecular structure at the nanoscale level in the intact cell, bridging the mesoscale gap to classical structural biology methodologies. Analysis of super-resolution data by artificial intelligence (AI), such as machine learning, offers tremendous potential for the discovery of new biology, that, by definition, is not known and lacks ground truth. Herein, we describe the application of weakly supervised paradigms to super-resolution microscopy and its potential to enable the accelerated exploration of the nanoscale architecture of subcellular macromolecules and organelles.

Figure 1.

AI and SRM. (A) Continued and parallel improvement in SRM hardware and AI-based analysis will lead to new and better approaches to define the organization and dynamics of subcellular structures. For instance, analysis of higher resolution single-molecule approaches, such as MinFlux (2 nm lateral resolution) (Balzarotti et al., 2017) with deep learning and self-supervision modalities should lead to novel insights into molecular structure in whole cell analyses. (B) Different levels of supervision include strong supervision in which each object or even each pixel has annotated information that can be leveraged. This is rare in SRM as discovery implies absence of such data. Weak supervision occurs when partial information, often at the image level, is available, such as the presence or absence of a type of object, but not its location. In SRM use cases, this would be cell line, treatment, gene expression etc. Self-supervision is a hybrid form, where the model learns to operate on images using indirect information, such as rotations. This is then followed by a fine-tuning stage with small amounts of strongly supervised data. Unsupervised, i.e., no supervision, indicates a complete absence of such metadata.

Figure 2.

Super-resolution microscopy and structural biology bridge the mesoscale domain. Structural biology approaches (NMR, X-ray crystallography, and cryoEM) with angstrom level resolution analyze structures as large as the 80–120 nm coronavirus. Whole-cell super-resolution microscopy approaches such as structured illumination (SIM), stimulated emission depletion (STED), single-molecule localization microscopy (SMLM), and MinFlux (images from Edrington et al., 2011; Gao et al., 2019; Ke et al., 2020; Khater et al., 2018; Rozov et al., 2019; Schmidt et al., 2021) enable whole cell analysis with increasing resolution broaching the nanoscale.

Figure 3.

Molecular architecture by SRM. The top row shows outer (left) and inner (right) structure views of the ArtScience Museum at Marina Bay Sands. Below, pixel-based representation of SRM data provides unprecedented high-resolution views of subcellular structures, such as caveolin-1 domains, (wide-field on left versus SMLM on right) or the endoplasmic reticulum (confocal in red versus STED in green) but remain analogous to enlarged, more detailed views of the external face, or outer structure, of buildings (inset). Ongoing AI-based semantic analysis of SRM will provide the means to explore the design basis, or molecular architecture, of subcellular macromolecules and organelles.

Figure 4.

The weakly supervised paradigm for AI-based semantic analysis of SRM datasets. In the absence of pixel or object level ground truth (strong supervision), prior biological knowledge defines group labels for weakly supervised training of SRM datasets to identify group-specific protein structures. Left: Caveolae expression is known to require CAV1 and the adaptor protein cavin-1 (Anderson, 1998; Hill et al., 2008). PC3 cells lack cavin-1 and therefore caveolae; native and Cavin-1 transfected PC3 cells therefore provided group labels for weakly supervised network analysis: SuperResNET. Using a proximity threshold that most significantly distinguished point clouds of these two biologically distinct groups, we then segmented clusters of points into blobs (structures or objects), based on 28 features describing size, shape, topology and network measures. Four groups of blobs were found in Cavin-1 expressing PC3 cells, one of which was significantly larger than either of the two groups found in PC3 cells, therefore corresponding to caveolae. Structural correspondence of identified structures to known biology, such as 8S complexes and caveolae whose structure has been determined by cryoEM (Han et al., 2020; Stoeber et al., 2016), validates the approach and enables identification of novel structures such as hemi-spherical CAV1 S2 scaffolds (Khater et al., 2018, 2019b). Right: HT-1080 and COS-7 cell lines differ in that only HT-1080 express elongated ribosome-studded mitochondria-ER contact sites (riboMERCS). Based on the differential expression of riboMERCs between these two cell lines, we developed a novel segmentation-free algorithm able to reconstruct MERCS, MCS-DETECT, that was optimized by maximizing the differential result between both cell lines (Cardoen et al., 2024). Feature analysis of MERCs matches differences between the two cell lines based on EM (i.e., ground truth), and the approach is validated by knockdown of a known riboMERC tether, RRBP1 (Hung et al., 2017). MCS-DETECT led to identification of extended tubular riboMERCs.