SCORCH: Improving structure-based virtual screening with machine learning classifiers, data augmentation, and uncertainty estimation

Abstract

Introduction: The discovery of a new drug is a costly and lengthy endeavour. The computational prediction of which small molecules can bind to a protein target can accelerate this process if the predictions are fast and accurate enough. Recent machine-learning scoring functions re-evaluate the output of molecular docking to achieve more accurate predictions. However, previous scoring functions were trained on crystalised protein-ligand complexes and datasets of decoys. The limited availability of crystal structures and biases in the decoy datasets can lower the performance of scoring functions.

Objectives: To address key limitations of previous scoring functions and thus improve the predictive performance of structure-based virtual screening.

Methods: A novel machine-learning scoring function was created, named SCORCH (Scoring COnsensus for RMSD-based Classification of Hits). To develop SCORCH, training data is augmented by considering multiple ligand poses and labelling poses based on their RMSD from the native pose. Decoy bias is addressed by generating property-matched decoys for each ligand and using the same methodology for preparing and docking decoys and ligands. A consensus of 3 different machine learning approaches is also used to improve performance.

Results: We find that multi-pose augmentation in SCORCH improves its docking power and screening power on independent benchmark datasets. SCORCH outperforms an equivalent scoring function trained on single poses, with a 1 % enrichment factor (EF) of 13.78 vs. 10.86 on 18 DEKOIS 2.0 targets and a mean native pose rank of 5.9 vs 30.4 on CSAR 2014. Additionally, SCORCH outperforms widely used scoring functions in virtual screening and pose prediction on independent benchmark datasets.

Conclusion: By rationally addressing key limitations of previous scoring functions, SCORCH improves the performance of virtual screening. SCORCH also provides an estimate of its uncertainty, which can help reduce the cost and time required for drug discovery.

Keywords: Docking; Drug discovery; Machine learning; Neural networks; Scoring; Virtual screening.

Graphical abstract

Fig. 1

Production workflow of single-pose and multiple-pose machine learning models. The single-pose dataset consisted of one pose for each protein-ligand pair and poses were labelled solely based on binding affinity. The multiple-pose dataset consisted of multiple poses for each protein ligand pair and poses were labelled based on binding affinity and RMSD from the native crystal pose. On each dataset, a GBDT (using XGBoost), a FF NN, and a W&D NN were trained and a consensus of the three model types was considered. Single-pose models were developed to evaluate whether using multiple poses with labelling stratified by both RMSD and binding affinity improves performance. SCORCH is a consensus of the three models trained on multiple-pose data. Additional details on data preparation (white boxes) are indicated in Figure S1.

Fig. 2

RMSD-based pose labelling example for PDB structure 1a0q. a) Docked poses (green) <2 Å from the crystal pose (thick blue lines) are given a label of 1. b) Docked poses (olive) between 2 Å and 4.5 Å from the crystal pose (thick blue lines) are excluded from the dataset. c) Docked poses (red) over 4.5 Å from the crystal pose (thick blue lines) are given a label of 0.

Fig. 3

Precision-Recall curves for produced machine-learning models and third-party scoring functions on 5,621 test set complexes.

Fig. 4

SCORCH is the best performing scoring function on a subset of the DEKOIS 2.0 independent benchmarking dataset. a) Precision-Recall curves for produced machine-learning models and third-party scoring functions on 18 DEKOIS 2.0 targets. b) Enrichment factors at 0.5 %, 1 %, 2 % and 5 % for produced machine-learning models and third-party scoring functions across 18 DEKOIS 2.0 targets. White diamonds indicate the mean EF.

Fig. 5

SCORCH certainty metric is a robust a priori indicator of enrichment success. a) Relationship between SCORCH certainty and residuals across all 22,319 DEKOIS 2.0 actives and decoys. Fitted regression line shown in red with the line equation displayed on the top left (p = 2e-16, P_r = −0.45); 95 % confidence intervals of the regression line are shown in grey. b) Relationship between enrichment factor and mean SCORCH certainty across top 0.5 % of ligands for 18 DEKOIS 2.0 receptors. Fitted regression line shown in red (p = 0.013, P_r = 0.57); 95 % confidence intervals of the regression line are shown in grey.

Fig. 6

RMSD-based pose labelling used in SCORCH improves the docking power compared to other MLSFs. a) Ranks of the lowest RMSD docked ligand pose across 510 test set complexes for produced machine-learning models and third-party scoring functions; White diamonds indicate the mean rank with the value displayed above each point. b) Ranks of the near native pose across the CSAR 2014 pose prediction benchmark for produced machine-learning models and third-party scoring functions. White diamonds indicate the mean rank with the value displayed above each point.

Fig. 7

10 features with highest influence on performance for six individual machine-learning models. Highly influential features for each model are listed on the y-axis. The x-axis represents the decrease in model validation set AUCPR when a given feature is replaced with random noise; a greater decrease indicates greater reliance on a given input feature for making predictions.