Reproducibility of Differential Proteomic Technologies in CPTAC Fractionated Xenografts

Abstract

The NCI Clinical Proteomic Tumor Analysis Consortium (CPTAC) employed a pair of reference xenograft proteomes for initial platform validation and ongoing quality control of its data collection for The Cancer Genome Atlas (TCGA) tumors. These two xenografts, representing basal and luminal-B human breast cancer, were fractionated and analyzed on six mass spectrometers in a total of 46 replicates divided between iTRAQ and label-free technologies, spanning a total of 1095 LC-MS/MS experiments. These data represent a unique opportunity to evaluate the stability of proteomic differentiation by mass spectrometry over many months of time for individual instruments or across instruments running dissimilar workflows. We evaluated iTRAQ reporter ions, label-free spectral counts, and label-free extracted ion chromatograms as strategies for data interpretation (source code is available from http://homepages.uc.edu/~wang2x7/Research.htm ). From these assessments, we found that differential genes from a single replicate were confirmed by other replicates on the same instrument from 61 to 93% of the time. When comparing across different instruments and quantitative technologies, using multiple replicates, differential genes were reproduced by other data sets from 67 to 99% of the time. Projecting gene differences to biological pathways and networks increased the degree of similarity. These overlaps send an encouraging message about the maturity of technologies for proteomic differentiation.

Keywords: CPTAC; Differential proteomics; iTRAQ; label-free; quality control; technology assessment; xenografts.

Figure 1

High-level view of the bioinformatics pipeline employed in this study. Six instruments analyzed the same xenograft pair. Label-free sets were processed once by a spectral counts method and once by extracted ion chromatograms. While instrument-specific assemblies of PSMs were used for repeatability analysis, an all-instrument assembly was analyzed for reproducibility and biological pathway and network enrichment.

Figure 2

Number of identified distinct peptides per sample/replicate. When data from each instrument were assembled separately with 0.5% PSM FDR and <5% empirical protein FDR, identification sensitivity varied considerably by site. Ailing instrument OElite@65 yielded the lowest sensitivity by far, whereas QExac@56 produced a remarkable number of identifications. Label-free instruments ran WHIM2 and WHIM16 separately, whereas iTRAQ instruments combined these samples into a single 4plex.

Figure 3

How many of the differential genes from each experiment are confirmed by at least one other replicate experiment? Blue differential genes are found in common by another experiment in this instrument, whereas orange ones are unique to a single replicate. For iTRAQ, the confirmation must come from a different set of LC–MS/MS data altogether. Note that instruments that produced more replicates were likely to have a higher proportion of common differential genes by random chance.

Figure 4

Rank correlations compare the ordering of genes by signed posterior probabilities. OVelos@10 illustrates the extent to which differential probabilities within an iTRAQ 4plex are more similar than those across multiple 4plexes. The QExac@56 shows an exception to this behavior in replicate B. Spectral count-based differentiation produces similar overall correlations to iTRAQ, though without the benefit of being able to compare within LC–MS/MS experiments. Replicate G from OVelos@65 correlated more poorly. The declining sensitivity of identification in QExac@98 gave low correlation values other than the one between the first two replicates.

Figure 5

To what extent are the differential genes found for each data set confirmed by other data sets? Those represented by blue were found to be differential in common with another data set in that graph panel, as well. Orange genes, on the other hand, were unique to a particular data set. High identification sensitivity for QExac@56 led to many instrument-specific differences.

Figure 6

NetGestalt-based analysis of functional consistency for up-in-WHIM16 genes from seven data sets. (a) Proteins in the iRef network are placed in a linear order together with the hierarchical modular organization of the network. Alternating green and orange bar colors distinguish neighboring modules. (b–g) For each data set, the network modules found to be enriched are colored light red. Within these modules, up-in-WHIM16 genes are represented by a dark red stripe.