Multi-site assessment of the precision and reproducibility of multiple reaction monitoring-based measurements of proteins in plasma

Abstract

Verification of candidate biomarkers relies upon specific, quantitative assays optimized for selective detection of target proteins, and is increasingly viewed as a critical step in the discovery pipeline that bridges unbiased biomarker discovery to preclinical validation. Although individual laboratories have demonstrated that multiple reaction monitoring (MRM) coupled with isotope dilution mass spectrometry can quantify candidate protein biomarkers in plasma, reproducibility and transferability of these assays between laboratories have not been demonstrated. We describe a multilaboratory study to assess reproducibility, recovery, linear dynamic range and limits of detection and quantification of multiplexed, MRM-based assays, conducted by NCI-CPTAC. Using common materials and standardized protocols, we demonstrate that these assays can be highly reproducible within and across laboratories and instrument platforms, and are sensitive to low mug/ml protein concentrations in unfractionated plasma. We provide data and benchmarks against which individual laboratories can compare their performance and evaluate new technologies for biomarker verification in plasma.

Figure 1

Sample preparation workflow for studies I, II and III. (a) Study I. Pooled, digested plasma was spiked with ¹²C and ¹³C/¹⁵N peptides to generate a nine-point standard curve. (b) Study II. An equimolar mixture of the seven target proteins was digested separately and spiked with an equimolar mixture of IS peptides. The digest of target proteins plus IS peptides was added to pooled, digested plasma. A nine-point standard curve was prepared with pooled, digested plasma spiked with an equimolar mixture of IS peptides as the diluent. Study I and study II samples were prepared centrally at NIST. (c) Study III. Undiluted plasma was spiked with an equimolar mixture of the target proteins, then diluted with plasma to generate a nine-point standard curve. Three aliquots of these samples (prepared at NIST) were then shipped to the eight participating sites where reduction, alkylation, digestion and desalting were carried out before SID-MRM-MS analysis. IS, internal standard; SPE, solid phase extraction.

Figure 2

Box plots of variation in MRM quantitative measurements, interlaboratory CV, intralaboratory CV and LOQ. (a) Intralaboratory assay CV. Box plots showing measured log concentration (y axis) versus theoretical (spiked-in) concentration (x axis) for HRP-SSD across the entire concentration range in diluted plasma. Protein concentration in µg/ml is mg protein equivalent in 1 ml undiluted plasma. The box plots for studies I and II are based on four replicate measurements, whereas those for study III summarize 12 measurements (four each from III a, b and c). Each of the eight sites was assigned a random numerical code (19, 52, 54, 56, 65, 73, 86, 95) for anonymization. (b) Interlaboratory assay CV. Values are shown for studies I–III for the entire range of HRP-SSD final analyte concentrations in plasma. Within each box plot, actual intralaboratory CV values for individual laboratories are shown with color-coded markers. The CV values are calculated based on the single best performing transition (lowest combined CV) across studies I and II. This same transition is also used for study III. (c) Interlaboratory assay LOQ. Values determined in studies I and II for the peptides indicated (see Table 1 for protein-peptide pair abbreviations). The inset values display the conversion of median LOQ to µg/ml (µg protein equivalent per 1 ml undiluted plasma) for each peptide. All measurements were made in 60-fold diluted plasma. Median is shown as a heavy horizontal line in all box plots. The box spans the interquartile range (IQR), with the whiskers extending to 1.5 × IQR. Values > 1.5 × IQR are deemed outliers, and shown as separate points.

Figure 3

Interlaboratory reproducibility of linear calibration curve slopes for study II. The eight plots display the concentration curves for the detection of HRP-SSD in study II across all laboratories. Each of the eight sites was assigned a random numerical code (19, 52, 54, 56, 65, 73, 86, 95) for anonymization. Comparison of the plots demonstrates good linearity, with the slopes falling close to the diagonal, black line (theoretical slope = 1), and good agreement between the three transitions at each concentration point. Four replicate measurements are represented at each concentration point. Analyte transitions: red diamond, transition 1, (m/z 492.6→703.4); blue asterisk, transition 2, (m/z 492.6→790.4); green triangle, transition 3, (m/z 492.6→974.5). In some cases, the data points overlay such that transition 1 is not visible. Inset plots show more detail of lower end of the concentration range. The mean slope calculation across all laboratories in this example is 0.794 with an interlaboratory CV of 18.7%. Final concentrations of heavy and light peptides and added proteins were adjusted according to the gravimetric measurements described in Supplementary Table 6a–f.

Figure 4

Response curves representing deviations from the trend line. Red diamond, transition 1; blue asterisk, transition 2; green triangle, transition 3. (a) Study I, site 52, MBP-HGF: interference in transition 1 of the analyte. (b) Study IIIb, site 95, MYO-LFT: interference in transition 2 of the analyte, which was also observed in study I, II and IIIa for this laboratory. (c) Study II, site 86, CRP-ESD: endogenous protein level increased the estimated protein concentration at the low end of the concentration range of spiked-in proteins, resulting in flattening of slope. (d) Study IIIa, site 56, LEP-IND: unstable electrospray conditions resulted in a substantial increase in interlaboratory CV to 99%. (e) Study IIIa, site 19, MBP-YLA: no detection of MBP-YLA peptide at any site. (f) Study I, site 86, PSA-IVG: saturation at highest two concentrations. Site codes are identical to those given in Figures 2 and 3.