High performance computing in biology: multimillion atom simulations of nanoscale systems

Abstract

Computational methods have been used in biology for sequence analysis (bioinformatics), all-atom simulation (molecular dynamics and quantum calculations), and more recently for modeling biological networks (systems biology). Of these three techniques, all-atom simulation is currently the most computationally demanding, in terms of compute load, communication speed, and memory load. Breakthroughs in electrostatic force calculation and dynamic load balancing have enabled molecular dynamics simulations of large biomolecular complexes. Here, we report simulation results for the ribosome, using approximately 2.64 million atoms, the largest all-atom biomolecular simulation published to date. Several other nano-scale systems with different numbers of atoms were studied to measure the performance of the NAMD molecular dynamics simulation program on the Los Alamos National Laboratory Q Machine. We demonstrate that multimillion atom systems represent a 'sweet spot' for the NAMD code on large supercomputers. NAMD displays an unprecedented 85% parallel scaling efficiency for the ribosome system on 1024 CPUs. We also review recent targeted molecular dynamics simulations of the ribosome that prove useful for studying conformational changes of this large biomolecular complex in atomic detail.

Figure 1

Increase in simulation system size with respect to year simulated. The largest bio-molecular sustainted performance simulations to date at the time of publication to our knowledge are shown. All simulations include explicit solvent unless otherwise noted. BPTI (VAC), bovine pancreatic trypsin inhibitor without solvent (McCammon 1977; Karplus and McCammon 2002); BPTI, bovine pancreatic trypsin inhibitor with solvent (Van Gunsteren and Karplus 1982); RHOD, photosynthetic reaction center of Rhodopseudomonas viridis (Heller et al. 1990); HIV-1, HIV-1 protease (Harte et al. 1992) ; ES, estrogen-DNA (Kosztin et al. 1997); STR, streptavidin (Eichinger et al. 1997); FN-III (Gao et al. 2002); DOPC, DOPC lipid bilayer (Tieleman 2004); RIBO, ribosome (Sanbonmatsu et al. 2005). Solid curve, Moore’s law doubling every 28.2 months. Dashed curve, Moore’s law doubling every 39.6 months. Dot-dashed curve, sinosoidal doubling fit (described in text).

Figure 2

Performance of NAMD on the LANL Q-Machine as a function of number of atoms. Solid symbols used a cutoff of 9 Å and dt = 2 fs with SHAKE. Open symbols used a cutoff of 12 Å and dt = 1 fs without SHAKE (Phillips, et al. parameters), resulting in a factor of ~2 increase in compute load and higher parallel efficiency but longer wall clock time per step. (a) Performance measured in GFLOP/s vs. number of processors. Performance increases with increasing system size. (b) Execution time per step as a function of the number of processors.

Figure 3

Parallel performance curve. Speed-up as a function of processors for systems with different numbers of atoms. Black curve represents ideal speed-up.

Figure 4

Physiological time simulated vs. number of processors for different numbers of atoms. The ‘turn-over’ in efficiency occurs between N_atoms = 5.73x10⁴ and 9.22x10⁴.

Figure 5

Performance vs. the number of atoms (black curves) and total number of atoms-ns simulated per day vs. number of atoms (red curves) for a constant number of processors (N_procs =512). Dashed curves with open symbols use Phillips, et al. parameters.

Figure 6

Memory usage vs. number of processors for different numbers of atoms. Simulations with N_atoms > 2x10⁶ require > 2 GB RAM per processor.

Figure 7

Solvation shell of the ribosome. Cyan, water density contours at ~ 3 times the bulk density, averaged over 1 ns. White = small subunit, Green = large subunit, Pink = mRNA, Red = aminoacyl-tRNA, Yellow = peptidyl-tRNA.

Figure 8

Aminoacyl-tRNA moves from the A/T state to the A/A state during the targeted molecular dynamics simulations. Blue, oxygen atom on every 5^th water molecule. White, 23S rRNA; light green, 50S ribosomal proteins; cyan, 16S rRNA; magenta, 30S ribosomal proteins; yellow, aminoacyl-tRNA; red, peptidyl-tRNA; green, mRNA. The top portion of the simulation domain is not shown in order to display the full tRNAs.

Figure 9

Entrance of the aminoacyl-tRNA 3’-CCA end (yellow) into the peptidyl transferase center of the large ribosomal subunit. Green, aminoacyl-tRNA amino acid; purple, 23S rRNA A-loop (LH92); pink, 23S rRNA LH90; blue, 23S rRNA LH89; red, universally conserved accommodation gate nucleotides; light green, peptidyl transferase center nucleotides that interact with the 3’-CCA end in the x-ray crystallography structure representing A/A state; cyan, peptidyl-tRNA amino acid.