Comparative Evaluation of MaxQuant and Proteome Discoverer MS1-Based Protein Quantification Tools

Abstract

MS1-based label-free quantification can compare precursor ion peaks across runs, allowing reproducible protein measurements. Among bioinformatic platforms enabling MS1-based quantification, MaxQuant (MQ) is one of the most used, while Proteome Discoverer (PD) has recently introduced the Minora tool. Here, we present a comparative evaluation of six MS1-based quantification methods available in MQ and PD. Intensity (MQ and PD) and area (PD only) of the precursor ion peaks were measured and then subjected or not to normalization. The six methods were applied to data sets simulating various differential proteomics scenarios and covering a wide range of protein abundance ratios and amounts. PD outperformed MQ in terms of quantification yield, dynamic range, and reproducibility, although neither platform reached a fully satisfactory quality of measurements at low-abundance ranges. PD methods including normalization were the most accurate in estimating the abundance ratio between groups and the most sensitive when comparing groups with a narrow abundance ratio; on the contrary, MQ methods generally reached slightly higher specificity, accuracy, and precision values. Moreover, we found that applying an optimized log ratio-based threshold can maximize specificity, accuracy, and precision. Taken together, these results can help researchers choose the most appropriate MS1-based protein quantification strategy for their studies.

Keywords: accuracy; differential analysis; label-free quantification; log ratio; mass spectrometry; precision; proteomics; sensitivity; specificity.

Figure 1

Schematic illustration of the experimental design of the study. The “Spike fmol” column reports the amount of spiked-in proteins contained in the different samples of the main data set, expressed in fmol. The “LR” column lists the protein abundance log ratio between groups expected for the spiked-in proteins in the six sample comparisons evaluated for the main data set.

Figure 2

Distribution of the number of spiked-in proteins identified (I) and quantified (Q) by MQ and PD in the nine samples of the main data set. The average number of identified/quantified proteins (N = 3 replicate runs) is reported for each sample.

Figure 3

Distribution of the number of spiked-in proteins quantified by MQ-L and/or PD in the nine samples of the main data set. The average number of quantified proteins (N = 3 replicate runs) is reported for each sample.

Figure 4

(A) Distribution of spiked-in proteins in classes based on their ρ values, calculated according to Spearman’s correlation between expected and measured quantitative values. Proteins with 14 or more MVs were classified in the “too many MVs” class. (B) Scatterplots of expected (x-axis) vs observed (y-axis) abundances and regression lines for the three proteins quantified in all sample replicates of the main data set with all quantification methods. Each plot displays the quantification values obtained according to a specific quantification method and reports the ρ value calculated for each protein.

Figure 5

Tukey’s boxplots showing the distribution of spiked-in protein abundance LRs obtained for the six comparisons performed within the main data set using the six quantification methods. The expected LR for each comparison is indicated by the dotted gray line. The LR value was calculated for proteins identified in all replicates of at least one sample group.

Figure 6

Statistical metrics of differential analysis results obtained from the six comparisons performed with the main data set. Five bar graphs are reported for each comparison. The first bar graph on the left illustrates the distribution of true positives (TPs), true negatives (TNs), false positives (FPs), false negatives (FNs), and proteins filtered out due to the high number of missing values (MVs) for the six quantification methods; spiked-in (left) and background (right) protein data are reported. The remaining four graphs, from left to right, show values of sensitivity, specificity, accuracy, and precision, respectively, reached by the six quantification methods.

Figure 7

Statistical metrics of differential analysis results obtained from the six comparisons performed with the main data set, upon application of an optimized LR-based threshold. The LR threshold was set to 1.4 for MQ-I, 1.0 for MQ-L, 1.6 for PD-I, 1.3 for PD-nI, 1.6 for PD-A, and 1.1 for PD-nA. The legends for the five bar graphs shown for each comparison are identical with those described for Figure 6.

Figure 8

Statistical metrics of differential analysis results obtained from the second data set. The legends for the five bar graphs are identical with those described for Figure 6.