Antibody-protein interactions: benchmark datasets and prediction tools evaluation

Abstract

Background: The ability to predict antibody binding sites (aka antigenic determinants or B-cell epitopes) for a given protein is a precursor to new vaccine design and diagnostics. Among the various methods of B-cell epitope identification X-ray crystallography is one of the most reliable methods. Using these experimental data computational methods exist for B-cell epitope prediction. As the number of structures of antibody-protein complexes grows, further interest in prediction methods using 3D structure is anticipated. This work aims to establish a benchmark for 3D structure-based epitope prediction methods.

Results: Two B-cell epitope benchmark datasets inferred from the 3D structures of antibody-protein complexes were defined. The first is a dataset of 62 representative 3D structures of protein antigens with inferred structural epitopes. The second is a dataset of 82 structures of antibody-protein complexes containing different structural epitopes. Using these datasets, eight web-servers developed for antibody and protein binding sites prediction have been evaluated. In no method did performance exceed a 40% precision and 46% recall. The values of the area under the receiver operating characteristic curve for the evaluated methods were about 0.6 for ConSurf, DiscoTope, and PPI-PRED methods and above 0.65 but not exceeding 0.70 for protein-protein docking methods when the best of the top ten models for the bound docking were considered; the remaining methods performed close to random. The benchmark datasets are included as a supplement to this paper.

Conclusion: It may be possible to improve epitope prediction methods through training on datasets which include only immune epitopes and through utilizing more features characterizing epitopes, for example, the evolutionary conservation score. Notwithstanding, overall poor performance may reflect the generality of antigenicity and hence the inability to decipher B-cell epitopes as an intrinsic feature of the protein. It is an open question as to whether ultimately discriminatory features can be found.

Figure 1

Flowchart for building benchmark datasets.

Figure 2

Hypothetical example of the structural alignment of proteins (A) (sequence AVCQYWC) and (B) (sequence ACYARTYC). Number of residue-residue matches = 5, number of residue-residue matches relative to the length the longest chain = 63% (5/8), sequence identity = 80% (4/5).

Figure 3

Two orthogonal views of a representative structure, influenza A virus hemagglutinin HA1 chain [PDB:1EO8]. Chain A is shown in light gray upon which are mapped epitope residues inferred from six protein structures in complexes with antibody fragments: HC45 Fab [PDB:1QFU] (blue), BH151 Fab [PDB:1EO8] (magenta), HC63 Fab [PDB:1KEN] (green), HC19 Fab [PDB:2VIR, 2VIS, 2VIT] (red). The hemagglutinin HA2 chain is shown in cyan. Residues common to HC45 and BH151 epitopes are shown in orange; residues common to HC63 and HC19 epitopes are shown in yellow; residue Tyr98 which is a part of HC19 epitope inferred from structure 2VIR but not from 2VIS and 2VIT structures is shown in black; The HC19 epitope residue Thr131 which is mutated to Ile in the 2VIS structure is shown in dark red. The HC19 epitope residue Thr155 which is mutated to Ile in 2VIT structure is shown in violet.

Figure 4

Distributions of DiscoTope scores for epitope, non-epitope and all protein residues.

Figure 5

Distribution of ConSurf scores for epitope and all protein residues. For the definition of confidence score see the Methods section.

Figure 6

Distribution of ProMate scores for epitope, non-epitope and all protein residues.

Figure 7

Distribution of PIER scores for epitope, non-epitope and all protein residues.

Figure 8

Average AUC values for each method. Vertical bars show one standard deviation.

Figure 9

Overall methods performance measured as average sensitivity and PPV values.

Figure 10

Proportion of successfully predicted epitopes.