A multi-laboratory study of diverse RSV neutralization assays indicates feasibility for harmonization with an international standard

Abstract

A current barrier to the standardized evaluation of respiratory syncytial virus (RSV) vaccine candidates is the wide variety of virus neutralization assay formats currently in use for assessing immunogenicity. Assay formats vary widely in labor intensiveness, duration, and sample throughput. Furthermore, the cell lines and virus strains used are not consistent among formats. The purpose of this multi-laboratory study was to assess the variability across a diverse array of assay formats that quantitate RSV neutralizing antibodies. Using a common specimen panel, the degree of overall agreement among existing assays was evaluated to inform on the need for harmonization of assay results. A total of 12 laboratories participated in the blinded survey study by testing a panel comprised of 57 samples chosen to span the reportable titer range of the assays. An independent statistical analysis was conducted to measure overall agreement of assay results. This analysis showed that precision was consistently high, whereas agreement varied widely among assays. To examine whether agreement could be improved, we conducted a harmonization exercise using a variety of sample types as pseudo standards. The results showed that the level of agreement could be improved, and provided information on the suitability of samples for developing an international standard.

Keywords: Antibodies; Assay; International standard; Neutralization; Respiratory syncytial virus; Vaccine.

Fig. 1

Box plots showing the distribution of neutralizing antibody titers measured for each sample by the laboratories. The box plots display the degree of agreement for each of the 57 samples in the specimen panel. In these box plots, the box is defined by the interquartile range (IQR) – the 25th and 75th percentiles of the distribution; the horizontal line within the box represents the median (50th percentile); and the asterisk represents the mean. Vertical lines extend to the most extreme observation that is less than 1.5 × the IQR, and the open diamonds correspond to moderate outliers (between 1.5–3.0 × IQR). There are no severe outliers (>3.0 × IQR; open squares). The 21 replicated natural infection (NI) serum samples are divided into low, medium, and high titer groups of 7 samples per group on the left-hand side of each plot. The 15 single (non-replicated) samples are grouped on the right hand side of the box plots figure.

Fig. 2

Within-laboratory and within-sample coefficients of variation. A: Within-lab coefficients of variation. In this graph, the closed circles correspond to the within-laboratory percentage coefficients of variation (CV) values for each of the 12 participating laboratories (one circle per lab). B: Within-sample coefficients of variation. In this graph, the closed circles correspond to the within-sample coefficients of variation (CV) values for the 21 duplicated samples in the specimen panel (one circle per duplicated sample).

Fig. 3

Scatter plot matrices illustrating the degree of agreement between labs before and after normalization. The scatter plot matrices display pairwise comparisons of RSV neutralizing antibody (nAb) titer values from the laboratories (A) before normalization and (B) after normalization using the purified RSV immune globulin (Ig) sample (RSVIg_10%) as the pseudo internal standard. In each scatter plot, each black dot plots the pairwise comparison of the log, base 2, antibody titers for a sample. The solid diagonal line indicates perfect agreement (intercept 0 and slope 1). The laboratories are designated by the blinded alphabetic identifications A-L.

Fig. 4

Graphs of statistical coefficient values showing the degree of agreement pre- and post-normalization. In each graph, the values of the statistical coefficients used to quantify precision (r), agreement (C_a), and overall agreement (r_c) are plotted in aggregate. Each open circle plots the value of the indicated statistical coefficient (r, C_a, or r_c) for the comparison of antibody titer results for all samples from a pair of laboratories. The values of these statistical coefficients prior to normalization are plotted in graph A. The values of these statistical coefficients after normalization using natural infection (NI) serum (sample NI_56M), purified RSV immune globulin (Ig) (sample RSVIg_10%), clinical trial sera (sample CT_5), and palivizumab (sample Pali_1mg) as pseudo internal standards for normalization are plotted in graphs B, C, D, and E, respectively.