Exploring the Influence of Experimental Design on Toxicity Outcomes in Zebrafish Embryo Tests

Abstract

Compound toxicity data obtained from independent zebrafish laboratories can vary vastly, complicating the use of zebrafish screening for regulatory decisions. Differences in the assay protocol parameters are the primary source of variability. We investigated this issue by utilizing data from the NTP DNT-DIVER database (https://doi.org/10.22427/NTP-DATA-002-00062-0001-0000-1, last accessed June 2, 2022), which consists of data from zebrafish developmental toxicity (devtox) and locomotor response (designated as "neurotox") screens from 3 independent laboratories, using the same set of 87 compounds. The data were analyzed using the benchmark concentration (BMC) modeling approach, which estimates the concentration of interest based on a predetermined response threshold. We compared the BMC results from 3 laboratories (A, B, C) in 3 toxicity outcome categories: mortality, cumulative devtox, and neurotox, in terms of activity calls and potency values. We found that for devtox screening, laboratories with similar/same protocol parameters (B vs C) had an active call concordance as high as 86% with negligible potency difference. For neurotox screening, active call concordances between paired laboratories are lower than devtox screening (highest 68%). When protocols with different protocol parameters were compared, the concordance dropped, and the potency shift was on average about 3.8-fold for the cumulative devtox outcome and 5.8-fold for the neurotox outcome. The potential contributing protocol parameters for potency shift are listed or ranked. This study provides a quantitative assessment of the source of variability in zebrafish screening protocols and sets the groundwork for the ongoing Systematic Evaluation of the Application of Zebrafish in Toxicology effort at the National Toxicology Program.

Keywords: computational toxicology; developmental neurotoxicity; developmental toxicity; zebrafish.

Figure 1.

Metrics to evaluate the response variation in wells treated only with vehicle control. The laboratory (Lab-A; Lab-B; Lab-C) with the lowest noise value/highest similarity in the evaluated metrics was sorted to the top: Lab-A is consistently to be the best in all evaluated metrics. A–C, Benchmark response (BMR) in 3 endpoint categories (mortality, cumulative devtox, and neurotox [similarity]). Each dot represents the BMR value (%) from an endpoint; 3 endpoints are available for neurotox(similarity) endpoints. D–F, Standard deviation (SD) values (%) in 3 endpoint categories. The SD values are calculated using 91/273/102 (Lab-A/Lab-B/Lab-C) responses across plates for mortality and cumulative devtox outcome category and 1432/2954/744 (Lab-A/Lab-B/Lab-C) responses across plates for neurotox outcome category. G, Median similarity of movement pattern between pairs of embryos when using Pearson’s correlation to evaluate the similarity of Light-dark (LD) movement pattern on 5 day-post- fertilization(dpf). Each dot represents the value from a plate. Boxplot was used to show the data distribution. The variability (represented as SD) is 0.07, 0.15, 0.11, for Lab-A, Lab-B, and Lab-C, respectively. The comparison of the remaining neurotox endpoints is available in the Supplementary Figure 3.

Figure 2.

The activity call concordance with confidence interval (CI) as the error bars in endpoints between 3 laboratories. The active/inactive concordance in endpoints between 4 types of comparisons: all (all 3 laboratories), Lab-A versus Lab-B, Lab-B versus Lab-C, and Lab-A versus Lab-C.

Figure 3.

The BMC difference of the concordantly active compounds between pairs of laboratories in 3 categories of endpoints (mortality, cumulative devtox, and neurotox). Each dot represents a compound and is colored based on its category (eg, drug). The violin plot is used to show the distribution of the dots. The 25th, 50th, and 75th percentiles of the distribution are shown as black lines in the violin. The red dashed line is equivalent to 0, meaning there is no potency difference. Compounds that are 10-fold more potent/less potent are labeled. The names in the cumulative devtox panel: 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD), dieldrin, aldicarb, dibenz(a, h)anthracene, pyrene, valinomycin, firemaster 550, dichlorodiphenyltrichloroethane (DDT), phenol, isopropylated, phosphate (3:1), and deltamethrin. A color version of this figure appears in the online version of this article.

Figure 4.

The fraction of variance explained by fixed/random effects in the LMM analyses. A, When fixed effect is the data source on 3 endpoint categories. B, For cumulative devtox and C, neurotox endpoint category, the fixed effect is one or a combination of the surrogate factors (DT_Factor1, NT_Factor1, and NT_Factor2) derived based on Table 3.