Critical assessment of methods of protein structure prediction (CASP)-Round XIV

Abstract

Critical assessment of structure prediction (CASP) is a community experiment to advance methods of computing three-dimensional protein structure from amino acid sequence. Core components are rigorous blind testing of methods and evaluation of the results by independent assessors. In the most recent experiment (CASP14), deep-learning methods from one research group consistently delivered computed structures rivaling the corresponding experimental ones in accuracy. In this sense, the results represent a solution to the classical protein-folding problem, at least for single proteins. The models have already been shown to be capable of providing solutions for problematic crystal structures, and there are broad implications for the rest of structural biology. Other research groups also substantially improved performance. Here, we describe these results and outline some of the many implications. Other related areas of CASP, including modeling of protein complexes, structure refinement, estimation of model accuracy, and prediction of inter-residue contacts and distances, are also described.

Keywords: Alphafold; CASP; community wide experiment; protein folding; protein structure prediction.

Figure 1:

Trend lines of backbone agreement with experiment for the best models in each of the 14 CASP rounds. Individual target points are shown for the most recent round. The three targets with the lowest agreement with experiment are colored blue (T1027 and T1029, NMR) and red (T0147s1, a subunit of a cryo-EM-derived heteromeric structure with complex inter-subunit interactions). The agreement metric, GDT_TS, is a multi-scale indicator of the closeness of the Cα atoms in a model to those in the corresponding experimental structure. Target difficulty is based on sequence and structure similarity to other proteins with known experimental structures. Performance in CASP14 (top black line) is very impressive, with accuracy approaching and in some cases likely exceeding experimental accuracy for many targets (see later text).

Figure 2:

Example of a high accuracy CASP14 model - CASP target T1053, a two-domain bacterial kinase. Model (from AlphaFold2, GDT_TS 93) in magenta, experimental structure (PDB 7m7a, resolution 3.2 Å) in turquoise. Both domains are difficult modeling targets (FM/TBM category).

Figure 3:

Superposition of a model (from AlphaFold2) of SARS CoV-2 ORF8 (CASP target T1064) and the corresponding experimental structure (PDB 7jtl, resolution 2.0 Å), illustrating the atomic level of agreement with experiment typically found in CASP14.

Figure 4:

(A) Average agreement of the best CASP14 models with experiment (GDT_TS) for different categories of experimental data. The first three bins show a fall-off as the resolution of X-ray structures decreases, suggesting lower GDT_TS values are partly due to higher experimental error. The effect is most pronounced for Cryo-EM experimental structures (right hand bin, resolution range 2.2 – 3.8 Angstroms). Two of the three NMR targets (not included here) have very low GDT_TS values (see text). (B) Distribution of experimental data type across categories of target difficulty. The large majority of targets in the most difficult category (FM) have low resolution X-ray, Cryo-EM (resolution range 2.2 – 3.8 Angstroms), or NMR data, whereas in the easiest category, 90% of targets are determined from higher resolution X-ray data.

Figure 5:

(A) Backbone agreement with experiment (GDT_TS) versus fraction of targets reaching a given level of agreement with experiment in different modeling difficulty categories. Trend lines for targets with the strongest homologous structural information available (‘TBM-easy’) are green, those where homology modeling is more difficult (‘TBM-Hard’) blue, those with only remote structural homologies (‘FM/TBM’) red, and the most difficult set with no detectable homology to known structures (‘FM’) black. Best models for each target. Targets with more information on homologous structures tend to be more accurate, but interpretation of that is complicated (see text). (B) Backbone agreement with experiment (Cα atom Root Mean Square Deviation) for different modeling difficulty categories (best models for each target).

Figure 6:

Best model backbone agreement with experiment (GDT_TS) as a function of log normalized sequence alignment depth (Neff/len) for targets with no detectable homology to known structures (‘Free Modeling’ (‘FM’)). Data for the most recent three CASPs. For this subset of the hardest ‘FM’ targets, dependence on alignment depth seen in earlier CASPs is not seen in CASP14.