LiveBench-2: large-scale automated evaluation of protein structure prediction servers

Abstract

The aim of LiveBench is to provide a continuous evaluation of structure prediction servers to inform developers and users about the current state-of-the-art structure prediction tools. LiveBench differs from other evaluation experiments because it is a large-scale and a fully automated procedure. Since LiveBench-1, which finished in April 2000, and related but independent CASP3 and CAFASP1 experiments, significant progress in the field has occurred. Some of the new developments have already been assessed at the recent CASP4 and CAFASP2 experiments (both independently of LiveBench), but others have not been observed yet because they entail developments carried out only recently. These include the availability of new servers (Pcons, FUGUE, and Coblath) and the enhancement of previously existing tools (mGenThreader, Sam-T, and 3D-PSSM), which illustrate the fast rate at which the field is advancing. Consequently, to keep in pace with the development, we present the results of the second large-scale evaluation of protein structure prediction servers. Of the 11 fold recognition servers evaluated, two servers appear to be most sensitive. One of these is 3D-PSSM, a server significantly improved after LiveBench-1. The other top performer is the new consensus server Pcons, which significantly outperformed other servers in the specificity of predictions. LiveBench-2 shows that the top performing servers are able to accurately recognize a fold for about one third of the "difficult" targets, a clear improvement over LiveBench-1 results. Given that automated structure prediction is increasingly becoming a biologists companion, the guidelines drawn from the LiveBench experiments are likely to provide users with valuable and timely information for their prediction needs.