MetSim: Integrated Programmatic Access and Pathway Management for Xenobiotic Metabolism Simulators

Abstract

Xenobiotic metabolism is a key consideration in evaluating the hazards and risks posed by environmental chemicals. A number of software tools exist that are capable of simulating metabolites, but each reports its predictions in a different format and with varying levels of detail. This makes comparing the performance and coverage of the tools a practical challenge. To address this shortcoming, we developed a metabolic simulation framework called MetSim, which comprises three main components. A graph-based schema was developed to allow metabolism information to be harmonized. The schema was implemented in MongoDB to store and retrieve metabolic graphs for subsequent analysis. MetSim currently includes an application programming interface for four metabolic simulators: BioTransformer, the OECD Toolbox, EPA's chemical transformation simulator (CTS), and tissue metabolism simulator (TIMES). Lastly, MetSim provides functions to help evaluate simulator performance for specific data sets. In this study, a set of 112 drugs with 432 reported metabolites were compiled, and predictions were made using the 4 simulators. Fifty-nine of the 112 drugs were taken from the Small Molecule Pathway Database, with the remainder sourced from the literature. The human models within BioTransformer and CTS (Phase I only) and the rat models within TIMES and the OECD Toolbox (Phase I only) were used to make predictions for the chemicals in the data set. The recall and precision (recall, precision) ranked in order of highest recall for each individual tool were CTS (0.54, 0.017), BioTransformer (0.50, 0.008), Toolbox in vitro (0.40, 0.144), TIMES in vivo (0.40, 0.133), Toolbox in vivo (0.40, 0.118), and TIMES in vitro (0.39, 0.128). Combining all of the model predictions together increased the overall recall (0.73, 0.008). MetSim enabled insights into the performance and coverage of in silico metabolic simulators to be more efficiently derived, which in turn should aid future efforts to evaluate other data sets.

Figure 1.

Workflow describing the main components of the study. Firstly a dataset was compiled, from which structural information was gathered. Predictions were generated in 4 tools making use of the MetSim framework to facilitate subsequent processing.

Figure 2.

Illustration of the hierarchical schema of a metabolic graph.

Figure 3.

Comparison of predicted metabolism map generated from TIMES against experimental metabolism map for Aripiprazole (DTXSID3046083) to illustrate how recall was derived.

Figure 4.

Hierarchically clustered heatmap of Recall clustered by chemical class for each metabolism simulator and model applied to the dataset. Model selections are indicated at the bottom of each column of the clustered heatmap as BioTransformer, CTS, Toolbox Vitro or Toolbox Vivo, and TIMES Vitro, TIMES Vivo or All Tools. Average recall for a given chemical class is illustrated by an increasingly dark gradient from zero (white) to unity (dark red) with the actual mean recall rate given in each cell. (Right) Bar chart of parent chemical class versus log2 scaled occurrence frequency in the dataset.

Figure 5.

ROC Curve showing the trade-off between TPR and FPR across the different simulators. TB(TIMES)_iv and TB(TIMES)_ivt refers to the Toolbox or TIMES in vivo and in vitro models.