An overview of the SAMPL8 host-guest binding challenge

Abstract

The SAMPL series of challenges aim to focus the community on specific modeling challenges, while testing and hopefully driving progress of computational methods to help guide pharmaceutical drug discovery. In this study, we report on the results of the SAMPL8 host-guest blind challenge for predicting absolute binding affinities. SAMPL8 focused on two host-guest datasets, one involving the cucurbituril CB8 (with a series of common drugs of abuse) and another involving two different Gibb deep-cavity cavitands. The latter dataset involved a previously featured deep cavity cavitand (TEMOA) as well as a new variant (TEETOA), both binding to a series of relatively rigid fragment-like guests. Challenge participants employed a reasonably wide variety of methods, though many of these were based on molecular simulations, and predictive accuracy was mixed. As in some previous SAMPL iterations (SAMPL6 and SAMPL7), we found that one approach to achieve greater accuracy was to apply empirical corrections to the binding free energy predictions, taking advantage of prior data on binding to these hosts. Another approach which performed well was a hybrid MD-based approach with reweighting to a force matched QM potential. In the cavitand challenge, an alchemical method using the AMOEBA-polarizable force field achieved the best success with RMSE less than 1 kcal/mol, while another alchemical approach (ATM/GAFF2-AM1BCC/TIP3P/HREM) had RMSE less than 1.75 kcal/mol. The work discussed here also highlights several important lessons; for example, retrospective studies of reference calculations demonstrate the sensitivity of predicted binding free energies to ethyl group sampling and/or guest starting pose, providing guidance to help improve future studies on these systems.

Keywords: Binding affinity; Blind challenge; Cucurbituril; Free energy; Host–guest binding; Octaacid; SAMPL.

Fig. 1

Structures of the CB8 host and drugs of abuse as guests molecules for SAMPL8. The barrel shaped macrocycle, CB8, host is shown on the top. It is composed of eight glycoluril units, and its carbonyl portal interacts and binds with cationic ammonium based guests inside the cavity. The guests for the SAMPL8 challenge are drugs of abuse (methamphetamine, fentanyl, morphine, hydromorphine, ketamine, PCP, and cocaine) which have the characteristics of typical CB[n] binders. The guests are named G1 through G7

Fig. 2

Structures of the GDCC hosts and guest molecules for the SAMPL8. (Top) TEMOA and TEETOA hosts in 2D and 3D representation. The 3D ball and stick figures are of TEMOA on the left and TEETOA on the right, (bottom) guests G1–5. The difference between the hosts is the functionality near the cavity opening, TEMOA has four methyl groups while TEETOA has four ethyl groups. The guests for SAMPL8 are named G1–5. Guests are fairly rigid molecules with carboxylate and/or hydroxyl groups. For TEMOA–G3, binding free energies were previously reported, thus was made optional for participants

Fig. 3

Protonation states considered for the CB8 guests. The figure shows protonation states considered at pH 7.4 for each guest in the CB8 dataset. All guests have at least two accessible protonation states, and guests G3 and G4 have three accessible states as predicted by Chemicalize from ChemAxon

Fig. 4

SAMPL8 host–guest experimental binding affinities. Experimental binding free energies for host–guest complexes in SAMPL8. All binding free energies ($\mathrm{\Delta }G$) were measured via ITC and/or NMR, and are represented as colored circles [CB8 (blue), TEMOA (yellow), and TEETOA (red)]. Experimental uncertainties representing the standard error of the mean are shown as black error bars in each circle

Fig. 5

CB8 error and correlation metrics for ranked methods. Shown here are violin plots of the distribution of performance for CB8. The error and correlation metrics (from top to bottom) include RMSE, MAE, R${}^2$, and $\tau$. The plots describe the shape of the distribution for each prediction in the dataset. For each error metric the median is indicated by a white circle in the violin plot. The black horizontal bars represent the first and third quartiles. The metrics and the relevant plots were generated by bootstrapping samples with replacement (including experimental uncertainties)

Fig. 6

CB8 correlation plots for ranked methods. Shown here are correlation plots comparing calculated versus experimental values for the DDM/FEP/MBAR/FM/RW[pm6s6], SILCS/LGFE/TIP3P/GCMC-MD, DDM/FEP/MBAR/FM/[mp2,b3lyp], DDM-SAMS/GAFF-DMBIS/TIP3P/MCMC-SAMS/, GFN2-xTB/MetaMD/GBSA/ensemble/Nobuffer, US/GAFF-AM1BCC/TIP3P/HRE-MD/emp_corr, GAFF-RESP/TIP3P/MD-Classical/xtb-GFN2B, MD/fmB3LYP(H)-fmMP2(G)/TIP3P/REUS/, ABFE/Parsley-GAFF-BCC/TIP3P/MD/NoBuffer1, and EE-MCC/GAFF2-AM1-BCC/TIP3P/MD ranked predictions for the CB8 dataset

Fig. 7

GDCC (TEMOA and TEETOA) error and correlation metrics for ranked methods. Shown here are violin plots of the distribution of performance for GDCC hosts. The error and correlation metrics (from top to bottom) include RMSE, MAE, R${}^2$, and $\tau$. The plots describe the shape of the distribution for each prediction in the dataset. For each error metric the median is indicated by a white circle in the violin plot. The black horizontal bars represent the first and third quartiles. The metrics and the relevant plots were generated by bootstrapping samples with replacement (including experimental uncertainties)

Fig. 8

RMSE and ME statistics by host–guest system for ranked methods. Shown here are the binding free energy error metrics by host–guest system (CB8, TEMOA, and TEETOA) across ranked methods. The root mean square error (RMSE) and mean signed error (ME) for $\mathrm{\Delta }G$ were computed via bootstrapping with replacement, which included experimental uncertainties, for all host–guest systems. This analysis includes all ranked methods submitted, but does not include predictions which were considered optional (CB8–G8, CB8–G9, and TEMOA–G3) in the analysis. The black error bars in the plots correspond to the 95-percentile bootstrap confidence intervals. Host–guest systems are color coded by host, where CB8 is represented by blue, TEMOA in yellow, and TEETOA in red

Fig. 9

TEETOA ethyl group preferred orientations are dependent on the orientation of the guest. A A simplified cartoon of the host–guest system (TEETOA–G1), highlighting the guest and orientation of four ethyl groups (EG1–4). The host is represented by a square which has symmetry of four. B Shows the actual conformation of the host–guest system cartoon shown in A. C–F Cartoons show preferred orientations of the four ethyl groups with different guest orientations in the host pocket. The host orientation is the same as A. A Represents the orientations of the four ethyl groups in panel B. Arrows in C–F show preferred orientations of the four ethyl groups (up: outward, down: inward)