Reproducibility of antimicrobial test methods

Abstract

We review reproducibility results for methods that test antimicrobial efficacy against biofilms, spores and bacteria dried onto a surface. Our review, that included test results for Pseudomonas aeruginosa, Salmonella choleraesuis and Bacillus subtilis, suggests that the level of reproducibility depends on the efficacy of the antimicrobial agent being tested for each microbe and microbial environment. To determine the reproducibility of a method, several laboratories must independently test the same antimicrobial agent using the method. Little variability among the efficacy results suggests good reproducibility. Such reproducibility assessments currently are hampered by the absence of an objective process for deciding whether the variability is sufficiently small. We present a quantitative decision process that objectively determines whether any method that assesses antimicrobial efficacy is reproducible. Because the perception of acceptable reproducibility may differ among stakeholders, the decision process is governed by a stakeholder's specifications that necessarily includes the efficacy of the agents to be tested.

Figure 1

Reproducibility of methods that test antimicrobials against bacteria dried onto a surface. The left pane shows results from a 7-lab study of the HSCT1 that tested agents for efficacy against S. choleraesuis dried onto a surface. The right panel shows results from a 8-lab study of the HSCT1, a 7-lab study of the HSCT2, and a 5-lab study of the UDM1 and UDM2 that tested products for efficacy against P. aeruginosa dried onto a surface. In both panes, each point corresponds to the reproducibility SD (S_R) and mean LR attained by a single product in the multi-laboratory study. The regression curves approximate S_R as a frown-shaped function of the mean LR for the HSCT1 and HSCT2. It is not possible to fit a regression curve to data from the UDM1 or UDM2 because only two agents were tested in the associated multi-laboratory studies.

Figure 2

Reproducibility of methods that test sporicides. Results are shown from multi-laboratory studies of 3 different methods that test agents against B. subtilis spores. The number of labs and products tested in each study are given in Table 1. The regression curves approximate the reproducibility SD (S_R) as a frown-shaped function of the mean LR for each method. Each point corresponds to S_R and the mean LR attained by a single agent in a multi-laboratory study.

Figure 3

Reproducibility of methods that test antimicrobials against biofilms. Results are shown from multi-laboratory studies of 3 different methods that test agents against P. aeruginosa biofilms. The number of labs and products tested in each study are given in Table 1. The regression curves approximate the reproducibility SD (S_R) as a frown-shaped function of the mean LR for each method. Each point corresponds to S_R and the mean LR attained by a single agent in a multi-laboratory study.

Figure 4

Assessing reproducibility of 3 antimicrobial test methods according to a stakeholder’s specifications. For a range of stakeholder specifications (δ = 1, 2 and 3; 0 < µ < 9; and γ = 90%) reproducibility assessments are provided for 3 of the methods in Table 1. The black curves that show the predicted reproducibility SDs (S_R) are from Figs 2,3. The curves depicting the maximum acceptable S_{R, max} were calculated from equation (1).

Figure 5

Assessing reproducibility of ANY antimicrobial test method according to a stakeholder’s specifications. For a range of stakeholder specifications (δ = 1, 2 and 3; γ = 90% and F(µ) = S_r²/S_R²), the maximum acceptable reproducibility SD of a method is determined, S_R,max (via equation (1)). There is 1 other input: the number of labs in the multi-laboratory study of a method.