Community-wide assessment of protein-interface modeling suggests improvements to design methodology

Abstract

The CAPRI (Critical Assessment of Predicted Interactions) and CASP (Critical Assessment of protein Structure Prediction) experiments have demonstrated the power of community-wide tests of methodology in assessing the current state of the art and spurring progress in the very challenging areas of protein docking and structure prediction. We sought to bring the power of community-wide experiments to bear on a very challenging protein design problem that provides a complementary but equally fundamental test of current understanding of protein-binding thermodynamics. We have generated a number of designed protein-protein interfaces with very favorable computed binding energies but which do not appear to be formed in experiments, suggesting that there may be important physical chemistry missing in the energy calculations. A total of 28 research groups took up the challenge of determining what is missing: we provided structures of 87 designed complexes and 120 naturally occurring complexes and asked participants to identify energetic contributions and/or structural features that distinguish between the two sets. The community found that electrostatics and solvation terms partially distinguish the designs from the natural complexes, largely due to the nonpolar character of the designed interactions. Beyond this polarity difference, the community found that the designed binding surfaces were, on average, structurally less embedded in the designed monomers, suggesting that backbone conformational rigidity at the designed surface is important for realization of the designed function. These results can be used to improve computational design strategies, but there is still much to be learned; for example, one designed complex, which does form in experiments, was classified by all metrics as a nonbinder.

Figure 1

Natural and designed complexes have similar overall properties. (A) buried surface area at the interface; (B) computed binding energy. Computed binding energies are reasonably correlated with experimentally determined dissociation constants (Pearson correlation r=0.53 ref. ). All plots were produced using gnuplot 4.4 and enhanced with Adobe Illustrator. In all figures, native refers to natural complexes in the docking benchmark.

Figure 2

Ability of different methods to discriminate between native and designed complexes. Receiver-operator Characteristic (ROC) curves are shown for each group, with the true- and false-positive classification on the y- and x-axes, respectively. The steeper the ascent of the curve and the larger the Area Under the Curve (AUC) the better the discrimination between natural and designed complexes. The green diagonal represents the expected output of random prediction. Percent AUC is noted within each plot. Groups 2 and 22 trained their metrics, in part, on Rosetta models published in the past, but not on the current set of designs (see Methods for more details).

Figure 3

Individual features that partially discriminate native and designed complexes. (A) Comparison of native and designs using the two most heavily weighted terms in the scoring function for each group. The points represent individual natives or designs, and the axes represent the most heavily weighted scoring terms. The scatter plots provide insight into some of the discriminatory power of the methods. While the phase-plane occupied by designs and natives overlap, in these cases, the designs occupy a small fraction of the plane with many of the natives having more favorable values. The results from Groups 11 and 33 suggest that the van der Waals contacts in designed interfaces are weaker than in natives. Likewise, Groups 6 and 11 suggest that solvation self energy (ACE) and electrostatics (the dominant contribution to Rosetta pair energy) are more optimized in natives. See individual groups' methods for more details. (B) Modification of the design protocol yields distributions of interface pairwise and Coulomb-electrostatic energies similar to those in natural complexes. Natural complexes (natives) and designs generated with (redesigns) and without (designs) an increased pairwise attractive term (weight=0.98) and Coulomb electrostatic interaction with a distance-dependent dielectric (weight=1.0). The distributions were calculated using pairwise attractive term and electrostatic interaction of 0.49 and 0.25, respectively, for all complexes. These designs have many flaws as potential binders, but can serve as decoys with more native-like distributions of electrostatic interactions.

Figure 4

Average number of neighbors (average degree) of interface residues within the designed monomer discriminates some designed complexes from native complexes. Surfaces with low average degree (bottom) tend to comprise segments, including unstructured regions, which are poorly embedded in the host monomer. By contrast, surfaces with high average degree (top) comprise secondary-structural elements and short loops that are better structurally connected to the host monomer. Following sequence design poorly connected surfaces might have altered conformations from those seen in the wild-type protein structure, providing some explanation for the failure of these designs to experimentally bind their targets. Average degree is marked on each panel. Clockwise from top-left, the panels represent designs 47, 59, 78, and 77 (coordinates are available in the online supplement). The target proteins are rendered in cyan. The backbones of the designed monomers are colored according to secondary structure (red – helix; yellow – strand; green – loop). Designed interfacial residues are shown in sticks with carbon, oxygen, and nitrogen, colored in green, red, and blue, respectively. Molecular representations were produced with PyMol.