Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Sep 19;3(9):11392-11406.
doi: 10.1021/acsomega.8b01647. eCollection 2018 Sep 30.

How Precise Are Our Quantitative Structure-Activity Relationship Derived Predictions for New Query Chemicals?

Affiliations

How Precise Are Our Quantitative Structure-Activity Relationship Derived Predictions for New Query Chemicals?

Kunal Roy et al. ACS Omega. .

Abstract

Quantitative structure-activity relationship (QSAR) models have long been used for making predictions and data gap filling in diverse fields including medicinal chemistry, predictive toxicology, environmental fate modeling, materials science, agricultural science, nanoscience, food science, and so forth. Usually a QSAR model is developed based on chemical information of a properly designed training set and corresponding experimental response data while the model is validated using one or more test set(s) for which the experimental response data are available. However, it is interesting to estimate the reliability of predictions when the model is applied to a completely new data set (true external set) even when the new data points are within applicability domain (AD) of the developed model. In the present study, we have categorized the quality of predictions for the test set or true external set into three groups (good, moderate, and bad) based on absolute prediction errors. Then, we have used three criteria [(a) mean absolute error of leave-one-out predictions for 10 most close training compounds for each query molecule; (b) AD in terms of similarity based on the standardization approach; and (c) proximity of the predicted value of the query compound to the mean training response] in different weighting schemes for making a composite score of predictions. It was found that using the most frequently appearing weighting scheme 0.5-0-0.5, the composite score-based categorization showed concordance with absolute prediction error-based categorization for more than 80% test data points while working with 5 different datasets with 15 models for each set derived in three different splitting techniques. These observations were also confirmed with true external sets for another four endpoints suggesting applicability of the scheme to judge the reliability of predictions for new datasets. The scheme has been implemented in a tool "Prediction Reliability Indicator" available at http://dtclab.webs.com/software-tools and http://teqip.jdvu.ac.in/QSAR_Tools/DTCLab/, and the tool is presently valid for multiple linear regression models only.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing financial interest.

Figures

Figure 1
Figure 1
Snapshot of the developed software Prediction Reliability Indicator.
Figure 2
Figure 2
Schematic diagram of workflow of the analysis.
Figure 3
Figure 3
Radar plots showing occurrence (in fractions of all cases) of weighting 0.5–0–0.5 for maximum % correct predictions for individual datasets and combined datasets.

References

    1. Dearden J. C. The History and Development of Quantitative Structure-Activity Relationships (QSARs). Int. J. Quant. Struct.-Prop. Relat. 2016, 1, 1–44. 10.4018/IJQSPR.2016010101. - DOI
    1. Roy K.; Kar S.; Das R. N.. Understanding the Basics of QSAR for Applications in Pharmaceutical Sciences and Risk Assessment; Academic Press: NY, 2015.
    1. Tropsha A. Best practices for QSAR model development, validation, and exploitation. Mol. Inf. 2010, 29, 476–488. 10.1002/minf.201000061. - DOI - PubMed
    1. Cherkasov A.; Muratov E. N.; Fourches D.; Varnek A.; Baskin I. I.; Cronin M.; Dearden J.; Gramatica P.; Martin Y. C.; Todeschini R.; Consonni V.; Kuz’min V. E.; Cramer R.; Benigni R.; Yang C.; Rathman J.; Terfloth L.; Gasteiger J.; Richard A.; Tropsha A. J. Med. Chem. 2016, 57, 4977–5010. 10.1021/jm4004285. - DOI - PMC - PubMed
    1. Roy K.; Mitra I. On various metrics used for validation of predictive QSAR models with applications in virtual screening and focused library design. Comb. Chem. High Throughput Screening 2011, 14, 450–474. 10.2174/138620711795767893. - DOI - PubMed