RECOVER identifies synergistic drug combinations in vitro through sequential model optimization

Abstract

For large libraries of small molecules, exhaustive combinatorial chemical screens become infeasible to perform when considering a range of disease models, assay conditions, and dose ranges. Deep learning models have achieved state-of-the-art results in silico for the prediction of synergy scores. However, databases of drug combinations are biased toward synergistic agents and results do not generalize out of distribution. During 5 rounds of experimentation, we employ sequential model optimization with a deep learning model to select drug combinations increasingly enriched for synergism and active against a cancer cell line-evaluating only ∼5% of the total search space. Moreover, we find that learned drug embeddings (using structural information) begin to reflect biological mechanisms. In silico benchmarking suggests search queries are ∼5-10× enriched for highly synergistic drug combinations by using sequential rounds of evaluation when compared with random selection or ∼3× when using a pretrained model.

Keywords: CP: Systems biology; active learning; deep learning; drug combination; drug synergy; in vitro screening; machine learning; oncology; sequential model optimization.

Graphical abstract

Figure 1

Overview of the RECOVER workflow integrating both a novel machine-learning pipeline and iterated wet-lab evaluation

Figure 2

Simulations suggest that RECOVER can enrich for highly synergistic combinations given a limited budget Reversed cumulative density of queried combinations following different querying strategies. (Inset) Level of enrichment. Shaded area corresponds to synergies $>54.9$. Results are averaged over 3 seeds.

Figure 3

Retrospective testing demonstrates the ability of RECOVER to generalize when at least one of the drugs has been seen during training but not beyond that (A) Overview of the different tasks on which RECOVER has been evaluated in preparation for the prospective evaluation within the preclinical framework. Each task corresponds to a different way to split the training, validation, and test sets and aims at evaluating a specific generalization ability of the model. (i.) Default. Combinations are split randomly into training/validation/test (70%/20%/10%). Only the MCF7 cell line is used. (ii.) One unseen drug. 30% of available drugs are excluded from the training and validation sets. The test set consists of combinations between a drug seen during training and an unseen drug. Combinations among seen drugs are split into training and validation (80%/20%). Only the MCF7 cell line is used. (iii.) Two unseen drugs. Similar to task (ii.), but the test set consists of combinations of two unseen drugs. (B and C) Performance of RECOVER and other models for the three different tasks. Standard deviation computed over 3 seeds.

Figure 4

In vitro evaluation demonstrates the significant enrichment for highly synergistic combinations through prospective use of RECOVER (A) Network plot indicating which pairs of drugs were identified at each round; line color and width represent synergy. (B) Heatmap representing drug combinations used during pretraining (NCI-ALMANAC), in the five subsequent rounds of experiments, and combinations excluded from the analysis. Drug combinations that were not available for pretraining or were not selected for experiments are represented in white. (C) Cumulative density plot of max Bliss synergy score for each experimental round; (inset) boxplot representation and calibration round details. (D) Predicted versus actual plot for max Bliss synergy score. The dotted line corresponds to $\mathrm{y}=\mathrm{x}$. (Inset) The explained variance is plotted for each experimental round. See also Data S1 and Tables S1 and S2.

Figure 5

RECOVER tends to map molecules with common biological mechanisms closely together (reflected by the similar colors of nearby points), even when structures are dissimilar UMAP of RECOVER drug embeddings with the color scheme generated to indicate the known target profile of the drugs; drugs that have molecular targets in common will have similar colors. Drug embeddings are learned using information from drug structures and viability screen data only.