Randomized SMILES strings improve the quality of molecular generative models

Abstract

Recurrent Neural Networks (RNNs) trained with a set of molecules represented as unique (canonical) SMILES strings, have shown the capacity to create large chemical spaces of valid and meaningful structures. Herein we perform an extensive benchmark on models trained with subsets of GDB-13 of different sizes (1 million, 10,000 and 1000), with different SMILES variants (canonical, randomized and DeepSMILES), with two different recurrent cell types (LSTM and GRU) and with different hyperparameter combinations. To guide the benchmarks new metrics were developed that define how well a model has generalized the training set. The generated chemical space is evaluated with respect to its uniformity, closedness and completeness. Results show that models that use LSTM cells trained with 1 million randomized SMILES, a non-unique molecular string representation, are able to generalize to larger chemical spaces than the other approaches and they represent more accurately the target chemical space. Specifically, a model was trained with randomized SMILES that was able to generate almost all molecules from GDB-13 with a quasi-uniform probability. Models trained with smaller samples show an even bigger improvement when trained with randomized SMILES models. Additionally, models were trained on molecules obtained from ChEMBL and illustrate again that training with randomized SMILES lead to models having a better representation of the drug-like chemical space. Namely, the model trained with randomized SMILES was able to generate at least double the amount of unique molecules with the same distribution of properties comparing to one trained with canonical SMILES.

Keywords: Chemical databases; Deep learning; Generative models; Randomized SMILES; Recurrent Neural Networks; SMILES.

Fig. 1

Traversal of the molecular graph of Aspirin using three methods: a the canonical ordering of the molecule; b atom order randomization without RDKit restrictions; c Atom order randomization with RDKit restrictions of the same atom ordering as b. Atom ordering is specified with a number ranking from 1 to 13 for each atom and the arrows show the molecular graph traversal process. Notice that the atom ordering is altered in c, prioritizing the sidechains (red arrows) when traversing a ring and preventing SMILES substrings like c1cc(c(cc1))

Fig. 2

Architecture of the RNN model used in this study. For every step $i$, input one-hot encoded token $X_i$ goes through an embedding layer of size $m\le w$, followed by $l>0$ GRU/LSTM layers of size $w$ with dropout in-between and then a linear layer that has dimensionality $w$ and the size of the vocabulary. Lastly a softmax is used to obtain the token probability distribution $Y_{\mathit{ij}}$. $H_i$ symbolizes the input hidden state matrix at step $i$

Fig. 3

Plot illustrating the percent of GDB-13 sampled alongside the sample size of the ideal model (blue) and the best of the canonical (yellow), randomized restricted (green) and randomized unrestricted (orange) models. Notice that the ideal model is always an upper bound and eventually ($n\sim 21B$) would sample the entire GDB-13. The trained models would reach the same point much later

Fig. 4

Histograms of different statistics from the randomized SMILES models. a Kernel Density Estimates (KDEs) of the number of randomized SMILES per molecule from a sample of 1 million molecules from GDB-13. The plot has the x axis cut at 5000, but the unrestricted randomized variant plot has outliers until 15,000. b KDEs of the molecule negative log-likelihood (NLL) for each molecule (summing the probabilities for each randomized SMILES) for the same sample of 1 million molecules from GDB-13. The plot is also cropped between range $\left(19,25\right)$. c Histograms between the NLL of all the restricted randomized SMILES of two molecules from GDB-13

Fig. 5

Linear regression plots between the UC-JSD and the UCC ratio. a Canonical SMILES $R^2=0.931$. b Restricted randomized SMILES $R^2=0.856$. c Unrestricted randomized SMILES $R^2=0.885$

Fig. 6

Kernel Density Estimates (KDEs) of the Molecule negative log-likelihoods (NLLs) of the ChEMBL models for the canonical SMILES variant (left) and the randomized SMILES variant (right). Each line symbolizes a different subset of 50,000 molecules from: Training set (green), validation set (orange), randomized SMILES model (blue) and canonical SMILES model (yellow). Notice that the Molecule NLLs for the randomized SMILES model (right) are obtained from the sum of all the probabilities of the randomized SMILES for each of the 50,000 molecules (adding up to 320 million randomized SMILES), whereas those from the canonical model are the canonical SMILES of the 50,000 molecules