Interlaboratory study characterizing a yeast performance standard for benchmarking LC-MS platform performance

Abstract

Optimal performance of LC-MS/MS platforms is critical to generating high quality proteomics data. Although individual laboratories have developed quality control samples, there is no widely available performance standard of biological complexity (and associated reference data sets) for benchmarking of platform performance for analysis of complex biological proteomes across different laboratories in the community. Individual preparations of the yeast Saccharomyces cerevisiae proteome have been used extensively by laboratories in the proteomics community to characterize LC-MS platform performance. The yeast proteome is uniquely attractive as a performance standard because it is the most extensively characterized complex biological proteome and the only one associated with several large scale studies estimating the abundance of all detectable proteins. In this study, we describe a standard operating protocol for large scale production of the yeast performance standard and offer aliquots to the community through the National Institute of Standards and Technology where the yeast proteome is under development as a certified reference material to meet the long term needs of the community. Using a series of metrics that characterize LC-MS performance, we provide a reference data set demonstrating typical performance of commonly used ion trap instrument platforms in expert laboratories; the results provide a basis for laboratories to benchmark their own performance, to improve upon current methods, and to evaluate new technologies. Additionally, we demonstrate how the yeast reference, spiked with human proteins, can be used to benchmark the power of proteomics platforms for detection of differentially expressed proteins at different levels of concentration in a complex matrix, thereby providing a metric to evaluate and minimize pre-analytical and analytical variation in comparative proteomics experiments.

Fig. 1.

Overview of analyses of yeast performance standard. Samples were processed centrally at NIST, and identical aliquots were distributed to the participating laboratories for LC-MS/MS analyses. Each sample was analyzed in triplicate, and the data were processed and analyzed centrally using a single analysis pipeline. For Study 6, each laboratory conformed to a prespecified SOP (supplemental Section C). For Study 8, no SOP was instituted, and the individual methods of each laboratory are described in supplemental Section D.

Fig. 2.

Detection efficiency of yeast proteins relative to their abundance. The gray curve shown in each panel denotes the TAP tag copy number distribution yeast proteins (derived from Ghaemmaghami et al. (16)); the abundance of proteins ranges from fewer than 50 to more than 10⁶ molecules per cell, and only 9% of yeast proteins have copy numbers greater than 20,000. Logistic regression curves (in color) indicate the probability of protein detection as a function of copy number for each run. The vertical lines indicate the mean copy number (for each instrument) corresponding to 50% probability of detection (CN50). Smaller CN50 values indicate greater depth of proteome sampling. The graph indicates that, on average, only the most abundant yeast proteins have a high probability of being detected in this one-dimensional LC-MS analysis. a shows the results for high protein loading (600 ng on column) when each lab uses their typical (non-SOP) testing protocol (Study 8). b shows the results for low protein loading (120 ng on column) using the same (non-SOP) protocol. c shows the results obtained from the SOP (Study 6; 120 ng on column). As expected, CN50 is increased in the low protein loading group compared with the high loading group (p < 0.0001). For equivalent “detectability,” a randomly selected protein must be present at nearly 40,200 copies per cell in the low loading group versus 24,650 copies per cell at high loading (95% confidence interval for the difference 11,150 to 20,470).

Fig. 3.

Detection efficiency of human proteins (UPS1) spiked into yeast background. This figure summarizes detection of UPS1 (supplemental Section E) and yeast proteins in the spiking experiments of Study 6. For all panels, the result of each RPLC run is indicated by a “+” plotting symbol; colors denote different instruments. Plotting symbols have been offset to avoid overlap of identical values. Protein detection is defined as observing two or more peptides mapping to the same protein (in a single RPLC run). On the x axis, “Spike concentration” refers to the concentration of the 48 equimolar human proteins (UPS1) spiked into the yeast matrix. Also on the x axis, “Yeast” refers to the unspiked matrix (i.e. 0 fmol/μl UPS1). a shows that the number of detected UPS1 proteins increases with increasing spike concentration. Note that at an equimolar spike concentration of 2.2 fmol/μl all instruments detect at least one UPS1 protein in each run. b and c show, respectively, the total number of yeast proteins and peptides detected per RPLC run. Different instruments show large variation in both the number of proteins/peptides detected and in the response to increasing spike concentration (p < 0.0001).