Comprehensive assessment of AlphaFold's predictions of secondary structure and solvent accessibility at the amino acid-level in eukaryotic, bacterial and archaeal proteins

Abstract

Numerous sequence-based predictors of the amino acid (AA)-level solvent accessibility (SA) and secondary structure (SS) of proteins have been developed. We empirically investigated whether these two key characteristics of AA-level structure can be accurately predicted from putative structures generated by the popular AlphaFold2. We compared AlphaFold2's results against several representative SS and SA predictors on a large test dataset that covers five distinct taxonomic groups (animals, plants, fungi, bacteria, and archaea). We used a broad collection of metrics that evaluate predictions of the numeric and binary (buried vs. solvent exposed) SA and the 3-state SS at both AA- and SS-region levels. We found that AlphaFold2 generated very accurate results, with high average Q₃ accuracy of 0.928 for the SS prediction and high Pearson Correlation Coefficient (PCC) of 0.815 between its putative and native SA values. AlphaFold2 significantly and consistently outperforms the considered predictors of SA and SS across the five taxonomic groups and both AA and region level evaluations. Moreover, we demonstrated that AlphaFold2 nearly perfectly reconstructs distributions of the sizes and numbers of the SS regions. We also showed that AlphaFold2 substantially improves over the SS and SA predictors when tested on a low sequence similarity test dataset, although its results and results of two other predictors suffer a modest drop in the quality of predicting SS regions. Altogether, our results suggest that AlphaFold2 makes very accurate predictions of SS and SA, which can be easily extracted from 200+ million pre-computed AF2's structure predictions in AlphaFoldDB.

Keywords: AlphaFold; Evaluation; Prediction; Protein structure; Secondary structure; Solvent accessibility.

Graphical abstract

Fig. 1

Assessment of the SA predictions at the AA level. The stick plots represent means and ranges of the assessment scores (MAE, PCC, MCC, and ACC) across different organisms (taxonomic groups). We showed results of the statistical significance analysis at the top of the plots where “* denote statistically significant differences with p-value < 0.05 when comparing a given predictor against AF2.

Fig. 2

Assessment of the SS predictions at the AA level. The stick plots represent means and ranges of the assessment scores (Q₃, Q_2H, Q_2C, Q_2E, and Q_error) across different organisms (taxonomic groups). We showed results of the statistical significance analysis at the top of the plots where “*” denote statistically significant differences with p-value < 0.05 when comparing a given predictor against AF2.

Fig. 3

Assessment of the SS predictions at the region level. The stick plots represent means and ranges of assessment scores (MAERs, SOVs, and KLs) across different organisms (taxonomic groups). We showed results of the statistical significance analysis at the top of the plots where “*” denote statistically significant differences with p-value < 0.05 when comparing a given predictor against AF2.