Statistical considerations for assessment of bioanalytical incurred sample reproducibility

Abstract

Bioanalytical method validation is generally conducted using standards and quality control (QC) samples which are prepared to be as similar as possible to the study samples (incurred samples) which are to be analyzed. However, there are a variety of circumstances in which the performance of a bioanalytical method when using standards and QCs may not adequately approximate that when using incurred samples. The objective of incurred sample reproducibility (ISR) testing is to demonstrate that a bioanalytical method will produce consistent results from study samples when re-analyzed on a separate occasion. The Third American Association of Pharmaceutical Scientists (AAPS)/Food and Drug Administration (FDA) Bioanalytical Workshop and subsequent workshops have led to widespread industry adoption of the so-called "4-6-20" rule for assessing incurred sample reproducibility (i.e. at least 66.7% of the re-analyzed incurred samples must agree within +/-20% of the original result), though the performance of this rule in the context of ISR testing has not yet been evaluated. This paper evaluates the performance of the 4-6-20 rule, provides general recommendations and guidance on appropriate experimental designs and sample sizes for ISR testing, discusses the impact of repeated ISR testing across multiple clinical studies, and proposes alternative acceptance criteria for ISR testing based on formal statistical methodology.

Fig. 1

Probability of failing ISR test versus number of analytical runs, for various . Sample size is 40 incurred samples. True total CV is 12% and true relative bias is 0%. Acceptance criteria based on the 4–6–20 rule

Fig. 2

Acceptance regions defined by 4–6–15 rule for QC samples and 4–6–20 rule for incurred samples. Dashed curve gives combinations of true relative bias and total CV such that the true proportion of QC sample concentrations within ±15% of nominal value is 66.7%. Solid curve gives combinations of true relative bias and total CV such that the true proportion of incurred sample repeat concentrations within ±20% of original concentration is 66.7%

Fig. 3

Probability of failing ISR test versus number of incurred samples, for true total CV = 15.5%, 17.5%, and 20.0%. True relative bias is 0%. Acceptance criteria based on the 4–6–20 rule

Fig. 4

Probability of failing ISR test versus number of incurred samples, for true total CV = 10.0%, 11.0%, and 12.0%. True relative bias is 0%. Acceptance criteria based on the 4–6–20 rule

Fig. 5

Probability of failing at least one ISR test versus number of ISR tests performed, for true total CV = 15.5%, 17.5%, and 20.0%. True relative bias is 0%. Sample size is 40 incurred samples per ISR test. Acceptance criteria based on the 4–6–20 rule

Fig. 6

Probability of failing at least one ISR test versus number of ISR tests performed, for true total CV = 10.0%, 11.0%, and 12.0%. True relative bias is 0%. Sample size is 40 incurred samples per ISR test. Acceptance criteria based on 4–6–20 rule

Fig. 7

Probability of failing ISR test versus number of incurred samples, for true total CV = 10.0%, 11.0%, and 12.0%. True relative bias is 0%. Acceptance criteria based on tolerance-interval approach

Fig. 8

Probability of failing ISR test versus number of incurred samples, for true total CV = 10.0%, 11.0%, and 12.0%. True relative bias is 0%. Acceptance criteria based on containment-proportion approach