Multilaboratory comparison of Streptococcus pneumoniae opsonophagocytic killing assays and their level of agreement for the determination of functional antibody activity in human reference sera

Abstract

Antibody-mediated killing of Streptococcus pneumoniae (pneumococcus) by phagocytes is an important mechanism of protection of the human host against pneumococcal infections. Measurement of opsonophagocytic antibodies by use of a standardized opsonophagocytic assay (OPA) is important for the evaluation of candidate vaccines and required for the licensure of new pneumococcal conjugate vaccine formulations. We assessed agreement among six laboratories that used their own optimized OPAs on a panel of 16 human reference sera for 13 pneumococcal serotypes. Consensus titers, estimated using an analysis-of-variance (ANOVA) mixed-effects model, provided a common reference for assessing agreement among these laboratories. Agreement was evaluated in terms of assay accuracy, reproducibility, repeatability, precision, and bias. We also reviewed four acceptance criterion intervals for assessing the comparability of protocols when assaying the same reference sera. The precision, accuracy, and concordance results among laboratories and the consensus titers revealed acceptable agreement. The results of this study indicate that the bioassays evaluated in this study are robust, and the resultant OPA values are reproducible for the determination of functional antibody titers specific to 13 pneumococcal serotypes when performed by laboratories using highly standardized but not identical assays. The statistical methodologies employed in this study may serve as a template for evaluating future multilaboratory studies.

FIG. 1.

Scatter plots of pairwise comparison between laboratories aggregated over serotype. Predicted OPA titers derived from ANOVA random-effects models were used for each set of assay duplicates. The solid line indicates perfect agreement (intercept 0 and slope 1).

FIG. 2.

Scatter plots of pairwise comparisons between laboratories and consensus values aggregated over serotypes. Predicted OPA titers for each laboratory and consensus titers were derived from ANOVA random-effects models. The solid line indicates perfect agreement (intercept 0 and slope 1).

FIG. 3.

Box plots by serotype and laboratory for the fold differences between the consensus and observed OPA titers. Consensus OPA titers were estimated for each sample within a serotype using the random-effects ANOVA model. In these plots, the box is defined by the 25th and 75th percentiles of the distribution; the horizontal line within the box represents the median or 50th percentile, and the asterisk signifies the mean. Vertical lines extend to the most extreme observation that is less than 1.5× the interquartile range (75th to 25th percentiles), and the diamonds and boxes correspond to moderate and severe outlying assay values, respectively.

FIG. 4.

Plots of within-laboratory (repeatability [▪]) and between-laboratory (reproducibility [•]) variability by serotype.