Are Protein Force Fields Getting Better? A Systematic Benchmark on 524 Diverse NMR Measurements

Abstract

Recent hardware and software advances have enabled simulation studies of protein systems on biophysically-relevant timescales, often revealing the need for improved force fields. Although early force field development was limited by the lack of direct comparisons between simulation and experiment, recent work from several labs has demonstrated direct calculation of NMR observables from protein simulations. Here we quantitatively evaluate recent molecular dynamics force fields against a suite of 524 chemical shift and J coupling ((3)JH(N)H(α), (3)JH(N)C(β), (3)JH(α)C', (3)JH(N)C', and (3)JH(α)N) measurements on dipeptides, tripeptides, tetra-alanine, and ubiquitin. Of the force fields examined (ff96, ff99, ff03, ff03*, ff03w, ff99sb*, ff99sb-ildn, ff99sb-ildn-phi, ff99sb-ildn-nmr, CHARMM27, OPLS-AA), two force fields (ff99sb-ildn-phi, ff99sb-ildn-nmr) combining recent side chain and backbone torsion modifications achieve high accuracy in our benchmark. For the two optimal force fields, the calculation error is comparable to the uncertainty in the experimental comparison. This observation suggests that extracting additional force field improvements from NMR data may require increased accuracy in J coupling and chemical shift prediction. To further investigate the limitations of current force fields, we also consider conformational populations of dipeptides, which were recently estimated using vibrational spectroscopy.

Figure 1

The overall χ² quantifies the agreement with all 524 experimental measurements.

Figure 2

For each class of model system, χ² quantifies the agreement between simulation and experiment.

Figure 3

The errors in ³J(H^NH^α) suggest that the ff99sb-ildn-phi and ff99sb-ildn-nmr force fields correct a significant bias in the ϕ potential of the ff99sb-ildn force field. Values are shown for TIP4P-EW.

Figure 4

Reduced χ² is shown for all 19 amino acids, indicating force field quality as a function of individual amino acid. Values are shown for TIP4P-EW with five well-performing force fields. Reduced χ² values near 1 indicate that force field error is comparable to the experimental uncertainty, while values much larger than 1 indicate possible room for force field improvements. Errors for reduced χ² are given by $\sqrt{\frac{2}{n}}$, where n is number of measurements available for that amino acid. Plotted error bars underestimate the true error, as error estimates include only the contribution of the Karplus and chemical shift prediction. This contribution tends to be the dominant source of error in the present benchmark.

Figure 5

The conformational populations for the 19 dipeptides (averaged over all 19) are shown for various force fields. Individual amino acid predictions are given in Figure S25. Grid ticks represent population increments of 0.1; the corners of the triangle represent the distributions with all β, α_R, and P_II, respectively. Also shown are experimental estimates and statistics from a PDB-derived coil library.