Hurdles and signposts on the road to virtual control groups-A case study illustrating the influence of anesthesia protocols on electrolyte levels in rats

Abstract

Introduction: Virtual Control Groups (VCGs) represent the concept of using historical control data from legacy animal studies to replace concurrent control group (CCG) animals. Based on the data curation and sharing activities of the Innovative Medicine Initiatives project eTRANSAFE (enhancing TRANSlational SAFEty Assessment through Integrative Knowledge Management) the ViCoG working group was established with the objectives of i) collecting suitable historical control data sets from preclinical toxicity studies, ii) evaluating statistical methodologies for building adequate and regulatory acceptable VCGs from historical control data, and iii) sharing those control-group data across multiple pharmaceutical companies. During the qualification process of VCGs a particular focus was put on the identification of hidden confounders in the data sets, which might impair the adequate matching of VCGs with the CCG. Methods: During our analyses we identified such a hidden confounder, namely, the choice of the anesthetic procedure used in animal experiments before blood withdrawal. Anesthesia using CO₂ may elevate the levels of some electrolytes such as calcium in blood, while the use of isoflurane is known to lower these values. Identification of such hidden confounders is particularly important if the underlying experimental information (e.g., on the anesthetic procedure) is not routinely recorded in the standard raw data files, such as SEND (Standard for Exchange of Non-clinical Data). We therefore analyzed how the replacement of CCGs with VCGs would affect the reproducibility of treatment-related findings regarding electrolyte values (potassium, calcium, sodium, and phosphate). The analyses were performed using a legacy rat systemic toxicity study consisting of a control and three treatment groups conducted according to pertinent OECD guidelines. In the report of this study treatment-related hypercalcemia was reported. The rats in this study were anesthetized with isoflurane. Results: Replacing the CCGs with VCGs derived from studies comprising both anesthetics resulted in a shift of control electrolyte parameters. Instead of the originally reported hypercalcemia the use of VCG led to fallacious conclusions of no observed effect or hypocalcemia. Discussion: Our study highlights the importance of a rigorous statistical analysis including the detection and elimination of hidden confounders prior to the implementation of the VCG concept.

Keywords: 3R; clinical chemistry; historical control data; replacement; systemic toxicity study; virtual control groups.

FIGURE 1

Data selection flow diagram.

FIGURE 2

Workflow of assessing the performance of VCGs.

FIGURE 3

(A) Histogram of the calcium values of the VCG sample population. (B) Boxplot of the VCG sample population. (C) Extracted original results of serum calcium concentration from the legacy study. If the mean difference between a dose group and the concurrent control exceeded the Dunnett-critical distance, the group was marked with an asterisk.

FIGURE 4

(A) Selection range of virtual control values and concurrent control values which are removed. (B) Mean values of the legacy study and the virtual control, colored with respect to whether the original statistical results were reproduced or not. If the mean difference between a dose group and the virtual control exceeded the Dunnett-critical distance, the group was marked with an asterisk.

FIGURE 5

(A) Calcium value distributions of male Wistar-rats (B) Box plots of these calcium levels with respect to the study year. The CO₂-group is colored grey, and the isoflurane-group is colored violet.

FIGURE 6

(A) Selection range of virtual control values along with concurrent control values which are removed. (B) Mean values of the legacy study and the virtual control, colored with respect to whether the original statistical results were reproduced or not. If the mean difference between a dose group and the virtual control exceeded the Dunnett-critical distance, the group was marked with an asterisk.

FIGURE 7

(A) Selection range of virtual control values along with concurrent control values which are removed and kept respectively. (B) Mean values of the legacy study and the virtual control, colored with respect to whether the original statistical results were reproduced or not. If the mean difference between a dose group and the virtual control exceeded the Dunnett-critical distance, the group was marked with an asterisk.

FIGURE 8

10 highest and 10 lowest calcium values of the Virtual Control Group (VCG) sample population (once with and once without the values affected by the confounder) compared to the Concurrent Control Group (CCG) and Dose group 3 of the legacy study. The 95% confidence interval for the difference in means is shown for each group.