Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct 15;43(27):1880-1886.
doi: 10.1002/jcc.26984. Epub 2022 Aug 24.

Conformer-RL: A deep reinforcement learning library for conformer generation

Runxuan Jiang  1 , Tarun Gogineni  1 , Joshua Kammeraad  2   3 , Yifei He  1 , Ambuj Tewari  1   2 , Paul M Zimmerman  3
Affiliations

Conformer-RL: A deep reinforcement learning library for conformer generation

Runxuan Jiang et al. J Comput Chem. .

Abstract

Conformer-RL is an open-source Python package for applying deep reinforcement learning (RL) to the task of generating a diverse set of low-energy conformations for a single molecule. The library features a simple interface to train a deep RL conformer generation model on any covalently bonded molecule or polymer, including most drug-like molecules. Under the hood, it implements state-of-the-art RL algorithms and graph neural network architectures tuned specifically for molecular structures. Conformer-RL is also a platform for researching new algorithms and neural network architectures for conformer generation, as the library contains modular class interfaces for RL environments and agents, allowing users to easily swap components with their own implementations. Additionally, it comes with tools to visualize and save generated conformers for further analysis. Conformer-RL is well-tested and thoroughly documented with tutorials for each of the functionalities mentioned above, and is available on PyPi and Github: https://github.com/ZimmermanGroup/conformer-rl.

Keywords: conformer generation; graph neural network; machine learning; reinforcement learning.

PubMed Disclaimer

Figures

FIGURE 1
FIGURE 1
Architecture of Conformer‐RL
FIGURE 2
FIGURE 2
Illustration of an agent interacting with the reinforcement learning environment in a conformer generation task for a simple molecule with two torsions. At iteration i, the environment state is the conformation of the molecule with each torsion at 0°. After the agent interacts with environment with the action [0,120], the environment sets the first torsion angle to 0° and the second torsion angle to 120°. The conformer is optimized using a molecular force field to get to the conformer state for the next iteration, where generally the angles will not be exactly equal to the action‐specified angles.
FIGURE 3
FIGURE 3
Example of using the toolkit to visualize a generated conformer in a Jupyter notebook
FIGURE 4
FIGURE 4
Analysis of conformers generated by conformer‐RL of lignin polymer with eight monomers. Highlighted are the proximity of reactive groups in the polymer, showing a distribution of interatomic distances for the Maccoll reaction mechanism
See this image and copyright information in PMC

References

    1. Silver D., Hubert T., Schrittwieser J., Antonoglou I., Lai M., Guez A., Lanctot M., Sifre L., Kumaran D., Graepel T., Lillicrap T., Simonyan K., Hassabis D., Science 2018, 362, 1140. - PubMed
    1. Vinyals O., Babuschkin I., Czarnecki W. M., Mathieu M., Dudzik A., Chung J., Choi D. H., Powell R., Ewalds T., Georgiev P., Oh J., Horgan D., M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard, D. Budden, Y. Sulsky, J. Molloy, T. L. Paine, C. Gulcehre, Z. Wang, T. Pfaff, Y. Wu, R. Ring, D. Yogatama, D. Wünsch, K. McKinney, O. Smith, T. Schaul, T. Lillicrap, Kavukcuoglu K., Hassabis D., Apps C., Silver D., Nature 2019, 575, 350. - PubMed
    1. Li Y., Kang H., Ye K., Yin S., Li X., Conference on Neural Information Processing Systems Deep Reinforcement Learning Workshop, Montréal, Canada, 2018. https://arxiv.org/abs/1812.00967
    1. Zhou Z., Li X., Zare R. N., ACS Cent. Sci. 2017, 3, 1337. - PMC - PubMed
    1. Simm G., Pinsler R., Hernandez‐Lobato J. M., in Proceedings of the 37th International Conference on Machine Learning, Vol. 119 (Eds: III H. D., Singh A.), PMLR, 2020, p. 8959 http://proceedings.mlr.press/v119/simm20b.html

Publication types