Pushing the boundaries of few-shot learning for low-data drug discovery with a Bayesian meta-learning hypernetwork framework

Abstract

Hunting for candidate compounds with favorable pharmacological, toxicological, and pharmacokinetic properties in drug discovery is essentially a low-data problem, as data acquisition is both challenging and costly. This inherent data limitation clashes with the requirements of many powerful deep learning models, which typically require large datasets. Here, we present Meta-Mol, a novel few-shot learning framework based on Bayesian Model-Agnostic Meta-Learning. Meta-Mol introduces a novel atom-bond graph isomorphism encoder that captures molecular structure information at the atomic and bond levels. This representation is further enhanced by a Bayesian meta-learning strategy, allowing for task-specific parameter adaptation and reducing overfitting risks. Additionally, a hypernetwork is employed to dynamically adjust weight updates across tasks, facilitating more complex posterior estimation. Our results demonstrate that Meta-Mol significantly outperforms existing models on several benchmarks, providing a robust solution to address data scarcity in drug discovery.

Keywords: Bayesian learning; few-shot learning; hypernetwork framework; meta-learning.

Figure 1

Conceptualization of Meta-Mol. (a) Advanced methods for molecular property prediction. (b) The MAML workflow in molecular property and activity prediction involves a process where the model learns a set of meta-parameters representing various related tasks based on existing ones and then rapidly adapts to new, unseen tasks using only a small amount of data. (c) Unlike previous approaches, Meta-Mol learns the universal weights point-wise, but a probabilistic structure is added when adapted for specific tasks.

Figure 2

Main architecture of Meta-Mol and Submodule Diagram. (a) Example of data representation for meta-learning-based molecular property prediction. (b) The overall architecture of Meta-Mol. (c) Diagram of dynamic sampling. Squares with different patterns represent tasks in the support set and query set, respectively. (d) Structure encoder module.

Figure 3

Comparative performance analysis diagram. (a) Performance evaluation of Meta-Mol and other representative methods on Tox21 and SIDER in 10-shot. (b) Loss curves for benchmark. The original data have been smoothed using a simple moving average with a window size of 40 to provide a clearer trend. (c) Model performance on ToxCast subsets. (d) Impact of different data proportions on model performance. The top-left section summarizes the trends of Meta-Mol across three datasets under various influencing factors, considering the PR-AUC metric. The other three radial sections compare the performance of different methods within the Tox21 dataset. (e) ROC-AUC performance across different shots. (f) Comparison of ROC-AUC and PR-AUC for various models on the MUV dataset.

Figure 4

A closer look at Meta-Mol. (a) Ablation study of Meta-Mol in 5-shot evaluation across five datasets. (b) Refined ablation study of Meta-Mol in a 5-shot scenario on the SIDER dataset. (c) Analysis of attention and parameter probability structures across six SIDER test tasks. Highlighted markers indicate whether the experimental result for a molecule in a given task is positive (1) or negative (0). (d) Visual analysis of molecular representations. The outputs of the Meta-Mol encoder are used as molecular representations, which are reduced in dimensionality using t-SNE for visualization. Different marker styles distinguish molecules with mixed, all-negative, or all-positive labels across the six test tasks. (e) Visual analysis of Meta-Mol’s performance (AUROC) across datasets. The rows represent source datasets used for training, and columns represent target datasets for fine-tuning. Full names for ToxCast subsets (e.g. ATG, BSK, CLD) are detailed in Table S4.