Target-decoy approach and false discovery rate: when things may go wrong

Abstract

The target-decoy approach (TDA) has done the field of proteomics a great service by filling in the need to estimate the false discovery rates (FDR) of peptide identifications. While TDA is often viewed as a universal solution to the problem of FDR evaluation, we argue that the time has come to critically re-examine TDA and to acknowledge not only its merits but also its demerits. We demonstrate that some popular MS/MS search tools are not TDA-compliant and that it is easy to develop a non-TDA compliant tool that outperforms all TDA-compliant tools. Since the distinction between TDA-compliant and non-TDA compliant tools remains elusive, we are concerned about a possible proliferation of non-TDA-compliant tools in the future (developed with the best intentions). We are also concerned that estimation of the FDR by TDA awkwardly depends on a virtual coin toss and argue that it is important to take the coin toss factor out of our estimation of the FDR. Since computing FDR via TDA suffers from various restrictions, we argue that TDA is not needed when accurate p-values of individual Peptide-Spectrum Matches are available.

Figure 1

Black box representation of an MS/MS search tool in target-decoy approach (TDA). (a) The search tool T takes as input the set of all spectra and the database of amino acid sequences (combination of the target and the decoy databases, but T cannot distinguish between them). T reports the list of peptides along with scores. The peptides that come from the target and the decoy databases are marked in green and red, respectively. (b) For each peptide identified by T (with score x), T⁺ simply adds an extra spurious peptide from the same protein (e.g., an overlapping peptide), assigning it to an arbitrarily chosen unidentified spectrum with the same parent mass and the same (fake) score x. Thus, T⁺ will double the number of peptide identifications in both target and decoy databases, for any score threshold. T⁺ is clearly a gimmick that we used for exposing vulnerability of TDA. We do not recommend using it in practice