Predicting in vivo effect levels for repeat-dose systemic toxicity using chemical, biological, kinetic and study covariates

Abstract

In an effort to address a major challenge in chemical safety assessment, alternative approaches for characterizing systemic effect levels, a predictive model was developed. Systemic effect levels were curated from ToxRefDB, HESS-DB and COSMOS-DB from numerous study types totaling 4379 in vivo studies for 1247 chemicals. Observed systemic effects in mammalian models are a complex function of chemical dynamics, kinetics, and inter- and intra-individual variability. To address this complex problem, systemic effect levels were modeled at the study-level by leveraging study covariates (e.g., study type, strain, administration route) in addition to multiple descriptor sets, including chemical (ToxPrint, PaDEL, and Physchem), biological (ToxCast), and kinetic descriptors. Using random forest modeling with cross-validation and external validation procedures, study-level covariates alone accounted for approximately 15% of the variance reducing the root mean squared error (RMSE) from 0.96 log₁₀ to 0.85 log₁₀ mg/kg/day, providing a baseline performance metric (lower expectation of model performance). A consensus model developed using a combination of study-level covariates, chemical, biological, and kinetic descriptors explained a total of 43% of the variance with an RMSE of 0.69 log₁₀ mg/kg/day. A benchmark model (upper expectation of model performance) was also developed with an RMSE of 0.5 log₁₀ mg/kg/day by incorporating study-level covariates and the mean effect level per chemical. To achieve a representative chemical-level prediction, the minimum study-level predicted and observed effect level per chemical were compared reducing the RMSE from 1.0 to 0.73 log₁₀ mg/kg/day, equivalent to 87% of predictions falling within an order-of-magnitude of the observed value. Although biological descriptors did not improve model performance, the final model was enriched for biological descriptors that indicated xenobiotic metabolism gene expression, oxidative stress, and cytotoxicity, demonstrating the importance of accounting for kinetics and non-specific bioactivity in predicting systemic effect levels. Herein, we generated an externally predictive model of systemic effect levels for use as a safety assessment tool and have generated forward predictions for over 30,000 chemicals.

Keywords: Effect levels; Predictive toxicity; Systemic effects.

Fig. 1

Schematic of the data preparation, model development, and model application workflow. “S” represents the study number per chemical and “C” represents the chemical index (for illustration purposes only)

Fig. 2

Histogram of study-level effect level (log₁₀ mg/kg/day) across 4379 animal toxicity studies. The overall effect level distribution constituted a mean of 1.7 log₁₀ mg/kg/day (~ 50 mg/kg/day) with a standard deviation of 0.94 log₁₀ mg/kg/day. ToxRefDB and HESS-DB had comparable mean effect level of 1.7 (blue dashed line) and 1.8 (green dashed line), respectively, whereas COSMOS had a mean effect level of 2.2 log₁₀ mg/kg/day (salmon dashed line)

Fig. 3

Boxplot of study-level effect level (log₁₀ mg/kg/day) stratified by individual covariate values. Continuous values were binned for presentation purposes (e.g., dose spacing). The upper and lower hinges (i.e., box) correspond to the first and third quartiles, while the upper and lower whiskers correspond to the highest and lowest values, respectively, within 1.5 of the inter-quartile range. Data beyond the whiskers are shown as dots. The over effect level median of 1.8 log₁₀ mg/kg/day is shown with the black vertical line

Fig. 4

Dotplot of variance explained (R ²) of cross-validation test sets for each bootstrap (n = 5) across five model sets including baseline and benchmark models. The five model sets were also run with varying chemical sets based on availability of biological and kinetic descriptors. The gray crossbar has been placed at the median R ² for each of the five model sets

Fig. 5

a Predicted vs observed study effect level (log₁₀ mg/kg/day) of the full external test set (N = 903 studies) using study-level covariates with chemical, biological and kinetic descriptors resulted in an R ² of 43% and an RMSE of 0.7 log₁₀ mg/kg/day. b Using the minimum predicted and observed effect level per chemical in the external test set (N = 249 chemicals), chemical effect level predictions were made and resulted in an R ² of 48% and an RMSE of 0.73 log₁₀ mg/kg/day