Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Dec 5;13(1):7499.
doi: 10.1038/s41467-022-35149-w.

Precise atom manipulation through deep reinforcement learning

Affiliations

Precise atom manipulation through deep reinforcement learning

I-Ju Chen et al. Nat Commun. .

Abstract

Atomic-scale manipulation in scanning tunneling microscopy has enabled the creation of quantum states of matter based on artificial structures and extreme miniaturization of computational circuitry based on individual atoms. The ability to autonomously arrange atomic structures with precision will enable the scaling up of nanoscale fabrication and expand the range of artificial structures hosting exotic quantum states. However, the a priori unknown manipulation parameters, the possibility of spontaneous tip apex changes, and the difficulty of modeling tip-atom interactions make it challenging to select manipulation parameters that can achieve atomic precision throughout extended operations. Here we use deep reinforcement learning (DRL) to control the real-world atom manipulation process. Several state-of-the-art reinforcement learning (RL) techniques are used jointly to boost data efficiency. The DRL agent learns to manipulate Ag adatoms on Ag(111) surfaces with optimal precision and is integrated with path planning algorithms to complete an autonomous atomic assembly system. The results demonstrate that state-of-the-art DRL can offer effective solutions to real-world challenges in nanofabrication and powerful approaches to increasingly complex scientific experiments at the atomic scale.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1. Atom manipulation with a DRL agent.
a The DRL agent learns to manipulate atoms precisely and efficiently through interacting with the STM environment. At each t, an action command at ~ π(st) is sampled from the DRL agent’s current policy π based on the current state st. The policy π is modeled as a multivariate Gaussian distribution with mean and covariance given by the policy neural network. The action at includes the conductance G, bias V, and the two-dimensional tip position at the start (end) of the manipulation xtip,start (xtip,end), which are used to move the STM tip to try to move the adatom to the target position. b The atom manipulation goal is to bring the adatom as close to the target position as possible. For Ag on Ag(111) surfaces, the fcc (face-centered cubic) and hcp (hexagonal close-packed) hollow sites are the most energetically favorable adsorption sites,. From the geometry of the adsorption sites, the error ε is limited to ranges from 0 nm to a3 depending on the target position. Therefore, the episode is considered successful and terminates if the ε is lower than a3. c STM image of an Ag adatom on Ag substrate. Bias voltage 1 V, current setpoint 500 pA.
Fig. 2
Fig. 2. DRL training results.
a, b The rolling mean (solid lines) and standard deviation (shaded areas) of episode reward, success rate, error, and episode length over 100 episodes showcase the training progress. The arrows indicate significant tip changes which occurred when the tip crashed deeply into the substrate and the tip apex needed to be reshaped to perform manipulation with the baseline parameters (see Methods) and the changes can be observed in the scan (see Supplementary Information). c The probability an atom is placed at the nearest adsorption site to the target at a given error P(xadatom = xnearestε) is calculated considering either only fcc sites or both fcc and hcp sites (see Methods section). With the error distribution of the 100 consecutive successful training episodes, we estimate the atoms are placed at the nearest site ~93% (only fcc sites) and ~61% (both fcc and hcp sites) of the time. d, e The DRL agent, which is continually trained, and the baseline are compared under three tip conditions that resulted from the tip changes indicated in a, b. The baseline uses bias V = 10 mV, conductance G = 6 μA/V, and tip movements illustrated in f. Under the three tip conditions, the baseline manipulation parameters lead to varying performances. In contrast, DRL always converges to near-optimal performances after sufficient continued training. f In the baseline manipulation parameter, the tip moves from the adatom position to the target position extended by 0.1 nm.
Fig. 3
Fig. 3. Atom manipulation statistics and autonomous construction of an artificial lattice.
a Top: Adatom movement distribution following manipulations visualized in a Gaussian kernel density estimation plot. Adatoms are shown to reside both on fcc and hcp hollow sites. Line-cuts in two directions r1 and r2 (indicated by the blue and red arrows) are shown in the bottom figure. b Atomically resolved point contact scan obtained by manipulating an Ag atom. Bias voltage 2 mV, current 74.5 nA. The lattice orientation is in good agreement with a. c Together with the assignment and path-planning algorithms, the trained DRL agent is used to construct an artificial 42-atom kagome lattice with atomic precision. Bias voltage 100 mV, current setpoint 500 pA.

References

    1. Eigler DM, Schweizer EK. Positioning single atoms with a scanning tunnelling microscope. Nature. 1990;344:524–526. doi: 10.1038/344524a0. - DOI
    1. Crommie MF, Lutz CP, Eigler DM. Confinement of electrons to quantum corrals on a metal surface. Science. 1993;262:218–220. doi: 10.1126/science.262.5131.218. - DOI - PubMed
    1. Moon CR, Lutz CP, Manoharan HC. Single-atom gating of quantum-state superpositions. Nat. Phys. 2008;4:454–458. doi: 10.1038/nphys930. - DOI
    1. Drost R, Ojanen T, Harju A, Liljeroth P. Topological states in engineered atomic lattices. Nat. Phys. 2017;13:668–671. doi: 10.1038/nphys4080. - DOI
    1. Kempkes SN, et al. Design and characterization of electrons in a fractal geometry. Nat. Phys. 2019;15:127–131. doi: 10.1038/s41567-018-0328-0. - DOI - PMC - PubMed