A synthetic peptide library for benchmarking crosslinking-mass spectrometry search engines for proteins and protein complexes

Abstract

Crosslinking-mass spectrometry (XL-MS) serves to identify interaction sites between proteins. Numerous search engines for crosslink identification exist, but lack of ground truth samples containing known crosslinks has precluded their systematic validation. Here we report on XL-MS data arising from measuring synthetic peptide libraries that provide the unique benefit of knowing which identified crosslinks are true and which are false. The data are analysed with the most frequently used search engines and the results filtered to an estimated false discovery rate of 5%. We find that the actual false crosslink identification rates range from 2.4 to 32%, depending on the analysis strategy employed. Furthermore, the use of MS-cleavable crosslinkers does not reduce the false discovery rate compared to non-cleavable crosslinkers. We anticipate that the datasets acquired during this research will further drive optimisation and development of XL-MS search engines, thereby advancing our understanding of vital biological interactions.

Fig. 1. Design of the crosslinked peptide library.

a the N-terminal amine group is protected from crosslinking with a biotin group that is followed by a linker region with a tryptic cleavage site. C-terminal lysine residues are incorporated with an azide group to prevent crosslinking to this site. b Peptides are crosslinked, treated with trypsin and the azide groups on the C-terminal lysine residues are reduced to an amine. Biotin-linker groups are removed from the solution with streptavidin beads. c The synthetic peptides are crosslinked in separate groups and combined prior to LC-MS analysis.

Fig. 2. Comparison of pLink, StavroX and Xi performance based on the crosslinked peptide library.

a number of CSMs that correspond to correct (black) and incorrect (grey) crosslinks identified by three search algorithms with incorporated FDR estimation. Results were filtered to an estimated 5% CSM-FDR, and the calculated FDR is given for each algorithm. b Agreement of correct (left) and incorrect CSMs (right) between pLink, StavroX and Xi for one technical repeat. c Number of correct unique crosslinks (black) and incorrect crosslinks (grey) identified with an estimated CSM-FDR of 5%. d Overlap of correct (left) and incorrect crosslinks (right) for one technical repeat. Values for figures a and c are given in Supplementary Tables 2 and 3, and average values ± standard deviation are shown on the stacked bar plots (rounded to the closest whole number). All results files can be found in Supplementary Data 1. Error bars correspond to the standard deviation between technical replicates (n = 3).

Fig. 3. Effect of crosslink validation strategies on overall results.

a Kojak results were validated by PeptideProphet or Percolator with which all CSMs were used, or only the top-scoring CSM for each species. b Xi results were validated at the CSM-FDR level, the unique CSM-FDR level or at the peptide pair-FDR level. c StavroX results were validated against a decoy database in which the sequences were shuffled with the protease sites remaining unchanged, or in which the sequences were inverted. All values are provided in Supplementary Table 6, and the average values ± standard deviation are shown on the stacked box plots. All results files can be found in Supplementary Data 1. Error bars correspond to the standard deviation between technical replicates (n = 3).

Fig. 4. Comparison of scores assigned to each CSM by pLink, StavroX, Xi and Kojak.

CSMs correlating to correct crosslinks are shown in black, and CSMs correlating to false positives are shown in red. a pLink vs. Xi, b pLink vs StavroX, c Xi vs StavroX, and d pLink vs Kojak.

Fig. 5. Score distributions for candidate and decoy crosslinks given by search algorithms.

a pLink, b Xi and c StavroX. Here, the true-positive and false-positive crosslinks are grouped as candidates. Green dashed line refers to the score cut-off used to filter for an estimated 5% FDR.

Fig. 6. number of MS2 spectra that are attributed to specific features.

a pLink, b StavroX and c Xi. Shown is the number of MS2 spectra annotated as CSMs, monolinks/looplinks/unmodified peptides, unvalidated PSMs with a score below the cut-off value for 5% FDR, decoy PSMs, and PSMs for which no information is known (unknown).

Fig. 7. Assessment of algorithms for the identification of peptides crosslinked with cleavable reagents DSBU and DSSO.

a, b Data were collected using MS2 method with stepped collision energy, analysed using MeroX (in Rise and RiseUP mode) and XlinkX, with data filtered to an estimated 5% FDR (a) and 1% FDR (b), with no additional score cut-off values employed. c Data for DSSO-crosslinked peptides were collected using HCD-, MS3- and ETD-based methods, and analysed using XlinkX with the recommended score cut-off values of 45 and 4 for crosslink score and Δcrosslink score, respectively. Values are given in Supplementary Tables 8–10.