Advancing atomic electron tomography with neural networks

Abstract

Accurate determination of three-dimensional (3D) atomic structures is crucial for understanding and controlling the properties of nanomaterials. Atomic electron tomography (AET) offers non-destructive atomic imaging with picometer-level precision, enabling the resolution of defects, interfaces, and strain fields in 3D, as well as the observation of dynamic structural evolution. However, reconstruction artifacts arising from geometric limitations and electron dose constraints can hinder reliable atomic structure determination. Recent progress has integrated deep learning, especially convolutional neural networks, into AET workflows to improve reconstruction fidelity. This review highlights recent advances in neural network-assisted AET, emphasizing its role in overcoming persistent challenges in 3D atomic imaging. By significantly enhancing the accuracy of both surface and bulk structural characterization, these methods are advancing the frontiers of nanoscience and enabling new opportunities in materials research and technology.

Keywords: 3D structural analysis; Atomic electron tomography; Neural network.

Fig. 1

Representative 3D atomic structural analysis based on electron tomography. (a) 3D positions of individual atoms in a tungsten needle sample. The 3D atomic structure consists of nine atomic layers along the [011] direction, labelled with dark red, red, orange, yellow, green, cyan, blue, magenta and purple from layers 1–9, respectively. Adapted from Reference [93], © 2015 Springer Nature. (b) A three-dimensional reconstruction of a Ag-Au nanocluster, showing atomic structure and composition of the cluster. Scale bar, 2 nm. Adapted from Reference [92], © 2015 Macmillan Publishers Limited. (c) 3D visualization of a reconstructed Au nanodecahedron containing more than 90 000 atoms. Adapted from Reference [91], this is an unofficial adaptation of an article that appeared in an ACS publication. ACS has not endorsed the content of this adaptation or the context of its use. © 2015 American Chemical Society. (d) Experimentally determined complex grain structure of an FePt nanoparticle via atomic electron tomography. Adapted from Reference [95], © 2017 Springer Nature. (e) Experimental 3D atomic model of an amorphous nanoparticle composed of eight chemical elements. Adapted from Reference [106] with permission, © 2021 Springer Nature. (f) 3D atomic models of an FePt nanoparticle after 9 min (left), 16 min (middle), and 26 min (right) of accumulated annealing. The top row shows the entire nanoparticle, while the bottom row highlights the Pt-rich core at each stage. Adapted from Reference [111], © 2019 Springer Nature. (g) 3D density maps and atomic positions of a single-crystalline Pt nanocrystal along the [111] zone axis. Scale bar, 1 nm. Adapted from Reference [110] with permission, © 2020 AAAS. (h-i) Experimentally determined 3D atomic structures of Pd@Pt core-shell nanoparticles, revealing (h) strain correlation between the surface and interface and (i) chemical diffusion at the interface. Adapted from References [103, 104]

Fig. 2

Neural network architectures for enhancing AET. (a) A residual-in-residual dense block-based GAN that fills missing regions in the sinogram domain as the first step of a two-stage process, followed by a U-Net-based network for artifact reduction in the reconstructed volume. Adapted from Reference [132]. (b) Architecture of the deep learning augmentation applied to 3D tomograms. The model follows a 3D U-Net structure, where each box represents a feature map. The number of channels is indicated below each feature map. Adapted from Reference [108]. (c) Overview of the skip fusing unit within the EC-UNETR framework. The outputs from the preceding subnetworks within the identical stage and the upsampled outputs from the lower stage are concatenated and normalized to calculate the fusing weights. Adapted from Reference [134] with permission, © 2024 IEEE

Fig. 3

Determination of Pt nanoparticle surface and interface structures via deep learning augmentation. (a-c) 3D iso-surfaces plotted with 10% iso-surface values (10% of the highest intensity), representing ground truth (a), linear tomogram before (b) and after the augmentation (c) from simulation of AET process for a Pt nanoparticle. Note that the z direction is the missing wedge direction. (d-f) 2-Å-thick slices perpendicular to [001] direction, obtained from the 3D tomograms near the center region. Ground truth (d), linear tomogram before (e) and after the augmentation (f). The grayscale background represents the reconstructed intensity, and blue dots represent the positions of traced atoms. Red circles denote misidentified atoms before the augmentation, which become correctly traced after the augmentation. Scale bar, 1 nm. (g) Experimentally determined 3D atomic structures of a Pt nanoparticle obtained via AET with neural network-based augmentation. The SiN substrate appears as black and gray disks. (h) Identified surface facets of the Pt nanoparticle, showing both < 100 > and < 111 > facets. Adapted from Reference [108]. (i) 3D structure of a Pt nanodumbbell revealing a twin boundary at the interface. Surface strain, measured at atomic resolution, was directly correlated with catalytic activity through DFT calculations. Adapted from Reference [60] with permission, © 2022 American Chemical Society

Fig. 4

Enhancing AET through image inpainting. (a) A representative denoised tilt-series image of a Pd nanoparticle supported on an Al₂O₃ substrate. (b) Al₂O₃ background extracted from (a) using a CNN-based image inpainting method. (c) Isolated Pd nanoparticle after background removal. Scale bar, 5 nm. (d-f) Reconstructed 3D atomic structure of the Pd nanoparticle viewed along (d) [100], (e) [101], and (f) [101] directions. The green and blue arrows indicate the tilt axis (y-axis) and the electron beam direction (z-axis), respectively. Adapted from Reference [135] with permission, © 2023 Wiley-VCH GmbH.