Avoiding a reproducibility crisis in regulatory toxicology-on the fundamental role of ring trials

Abstract

The ongoing transition from chemical hazard and risk assessment based on animal studies to assessment relying mostly on non-animal data, requires a multitude of novel experimental methods, and this means that guidance on the validation and standardisation of test methods intended for international applicability and acceptance, needs to be updated. These so-called new approach methodologies (NAMs) must be applicable to the chemical regulatory domain and provide reliable data which are relevant to hazard and risk assessment. Confidence in and use of NAMs will depend on their reliability and relevance, and both are thoroughly assessed by validation. Validation is, however, a time- and resource-demanding process. As updates on validation guidance are conducted, the valuable components must be kept: Reliable data are and will remain fundamental. In 2016, the scientific community was made aware of the general crisis in scientific reproducibility-validated methods must not fall into this. In this commentary, we emphasize the central importance of ring trials in the validation of experimental methods. Ring trials are sometimes considered to be a major hold-up with little value added to the validation. Here, we clarify that ring trials are indispensable to demonstrate the robustness and reproducibility of a new method. Further, that methods do fail in method transfer and ring trials due to different stumbling blocks, but these provide learnings to ensure the robustness of new methods. At the same time, we identify what it would take to perform ring trials more efficiently, and how ring trials fit into the much-needed update to the guidance on the validation of NAMs.

Keywords: OECD Test Guidelines; Reliability; Ring trials; Robustness; Validation.

Fig. 1

A schematic representation of a test method, its components, and its performance properties. (redrawn and modified from Worth and Balls 2001)

Fig. 2

Modular approach of the validation process. (Redrawn from Hartung et al. 2004)

Fig. 3

Steps from method development to method validation and finally to its regulatory use. Some aspects, like the toxicological and regulatory relevance, are considered at the first point of assessing a new method’s readiness to go into validation