DL_FFLUX: A Parallel, Quantum Chemical Topology Force Field

Abstract

DL_FFLUX is a force field based on quantum chemical topology that can perform molecular dynamics for flexible molecules endowed with polarizable atomic multipole moments (up to hexadecapole). Using the machine learning method kriging (aka Gaussian process regression), DL_FFLUX has access to atomic properties (energy, charge, dipole moment, etc.) with quantum mechanical accuracy. Newly optimized and parallelized using domain decomposition Message Passing Interface (MPI), DL_FFLUX is now able to deliver this rigorous methodology at scale while still in reasonable time frames. DL_FFLUX is delivered as an add-on to the widely distributed molecular dynamics code DL_POLY 4.08. For the systems studied here (10³-10⁵ atoms), DL_FFLUX is shown to add minimal computational cost to the standard DL_POLY package. In fact, the optimization of the electrostatics in DL_FFLUX means that, when high-rank multipole moments are enabled, DL_FFLUX is up to 1.25× faster than standard DL_POLY. The parallel DL_FFLUX preserves the quality of the scaling of MPI implementation in standard DL_POLY. For the first time, it is feasible to use the full capability of DL_FFLUX to study systems that are large enough to be of real-world interest. For example, a fully flexible, high-rank polarized (up to and including quadrupole moments) 1 ns simulation of a system of 10 125 atoms (3375 water molecules) takes 30 h (wall time) on 18 cores.

Figure 1

Gradient paths for a water dimer. The circles represent nuclei (attractors). Thick black lines are interatomic surfaces, and the squares that lie on these lines are so-called bond critical points. The second hydrogen of the left water is perpendicular to the plotting plane and not shown.

Figure 2

Two-dimensional simulation cell split into two domains. The dashed line represents the divide between the domains. The halo of P = 0 is shown by the shaded area.

Figure 3

Two-dimensional simulation cell split into two domains. The full halo of P = 0 is shown by the shaded area, accounting for PBCs.

Figure 4

Flow diagram demonstrating the stages of a typical DL_FFLUX MD time step.

Figure 5

Pseudo code for DL_FFLUX IQA energy and force calculation loop before modification.

Figure 6

Pseudo code for DL_FFLUX IQA energy and force calculation loop after modification.

Figure 7

Original charge prediction loop.

Figure 8

Modified charge prediction loop.

Figure 9

Profiles of the un-optimized (left) and optimized (right) code broken down into DL_FFLUX and DL_POLY contributions. The charts are not to scale, the right-hand-side chart represents a computation that is ∼75 000× faster than the left-hand-side chart.

Figure 10

Strong scaling, both actual and perfect, on log–log axes for all four water boxes: (top left) 5184 atoms in a (38 Å)³ box; (top right) 10 125 atoms in a (47 Å)³ box; (bottom left) 46 875 atoms in a (78 Å)³ box; and (bottom right) 107 811 atoms in a (103.5 Å)³ box.

Figure 11

Speed-up of the whole code relative to serial for the 107 811-atom water box.

Figure 12

Speed-up of the DL_FFLUX prediction routine only relative to serial for the 107 811-atom water box.

Figure 13

Strong scaling on multiple nodes for the 107 811-atom system. Log–log scaling (left) and speed-up relative to serial (right).

Figure 14

Time for a DL_FFLUX simulation divided by the time for a DL_POLY simulation with a flexible SPC water potential using the 107 811-atom water box. Averaged over all values of N_p tested.

Figure 15

Speed-up, relative to serial, of DL_FFLUX and DL_POLY routines at L′ = 0 (left) and L′ = 2 (right).

Figure 16

Profiles of the code at L′ = 0 (top row) and L′ = 2 (bottom row). Breakdowns are shown for N_p = 1, 8, and 36 going from left to right. The charts are not to scale, as N_p increases the total time taken decreases.

Figure 17

Timings at the optimum number of MPI processes for each system in nanoseconds per day.