Molecular graph convolutions: moving beyond fingerprints

Abstract

Molecular "fingerprints" encoding structural information are the workhorse of cheminformatics and machine learning in drug discovery applications. However, fingerprint representations necessarily emphasize particular aspects of the molecular structure while ignoring others, rather than allowing the model to make data-driven decisions. We describe molecular graph convolutions, a machine learning architecture for learning from undirected graphs, specifically small molecules. Graph convolutions use a simple encoding of the molecular graph-atoms, bonds, distances, etc.-which allows the model to take greater advantage of information in the graph structure. Although graph convolutions do not outperform all fingerprint-based methods, they (along with other graph-based methods) represent a new paradigm in ligand-based virtual screening with exciting opportunities for future improvement.

Keywords: Artificial neural networks; Deep learning; Machine learning; Molecular descriptors; Virtual screening.

Fig. 1

Molecular graph for ibuprofen. Unmarked vertices represent carbon atoms, and bond order is indicated by the number of lines used for each edge

Fig. 2

P → A operation. P^x is a matrix containing features for atom pairs ab, ac, ad, etc. The v_i are intermediate values obtained by applying f to features for a given atom pair. Applying g to the intermediate representations for all atom pairs involving a given atom (e.g. a) results in a new atom feature vector for that atom

Fig. 3

A → P operation. A^x is a matrix containing features for atoms a, b, etc. The v_i are intermediate values obtained by applying f to features for a given pair of atoms concatenated in both possible orderings (ab and ba). Applying g to these intermediate ordered pair features results in an order-independent feature vector for atom pair ab

Fig. 4

Weave module. This module takes matrices A^k and P^k (containing atom and pair features, respectively) and combines A → A, P → P, P → A, and A → P operations to yield a new set of atom and pair features (A^k+1 and P^k+1, respectively). The output atom and pair features can be used as input to a subsequent Weave module, which allows these modules to be stacked in series to an arbitrary depth

Fig. 5

Fuzzy histogram with three Gaussian “bins”. Each curve represents the membership function for a different bin, indicating the degree to which a point contributes to that bin. The vertical blue line represents an example point which contributes normalized densities of <0:01, ~0:25, and ~0:75 to the bins (from left to right)

Fig. 6

Abstract graph convolution architecture. In the current implementation, only the final atom features are used to generate molecule-level features

Fig. 7

Comparison of models with “simple” and “full” input featurizations. The simple featurization only encodes atom type, bond type, and graph distance. The full featurization includes additional features such as aromaticity and hydrogen bonding propensity (see “Molecule-level features” section for more details). Confidence intervals for box plot medians were computed as $\pm 1.57\times \mathrm{IQR}∕\sqrt{\mathrm{N}}$ [20]

Fig. 8

Graph convolution feature evolution. Atoms or pairs are displayed on the y-axis and the dimensions of the feature vectors are on the x-axis. a Conversion of the molecular graph for ibuprofen into atom and (unique) atom pair features. b Evolution of atom features after successive Weave modules in a graph convolution model with a W₃N₂ architecture and depth 50 convolutions in Weave modules. c Evolution of “simple” atom features (see “Input featurization” section) starting from initial encoding and progressing through the Weave modules of a W₂N₂ architecture. The color bar applies to all panels

Fig. 9

Comparison of models with different numbers of Weave modules with a model containing a single Weave module. All models used a maximum atom pair distance of two. The y-axis is cropped to emphasize differences near zero

Fig. 10

Comparison of root-mean-square (RMS) and Gaussian histogram reductions versus sum reduction. The y-axis reports difference in fivefold mean AUC relative to sum reduction. All models used two Weave modules and a maximum atom pair distance of two. The y-axis is cropped to emphasize differences near zero

Fig. 11

Comparison of models with different maximum atom pair distances to a model with a maximum pair distance of one (bonded atoms). All models have two Weave modules. The y-axis is cropped to emphasize differences near zero