Petascale Quantum Simulations of Nano Systems and Biomolecules

Size: px

Start display at page:

Download "Petascale Quantum Simulations of Nano Systems and Biomolecules"

Alfred King
5 years ago
Views:

Petascale Quantum Simulations of Nano Systems and Biomolecules J. Bernholc, E. Briggs, W. Lu,Y. Li and M. Hodak North Carolina State University, Raleigh I.

1 Petascale Quantum Simulations of Nano Systems and Biomolecules J. Bernholc, E. Briggs, W. Lu,Y. Li and M. Hodak North Carolina State University, Raleigh I. RMG petascale, open-source electronic structure code Blue Waters community Portal Part of Sustained Petascale Performance benchmark II. Si nanowire-based sensors for potential DNA sequencing ~12k atom DFT quantum transport calculations Sensitivity versus length & cross section Sensitivity enhancement via gate potential III. Nanoscale transistors for post-moore s era Multi-terminal quantum transport Quantum interference and tunneling leakage NC STATE UNIVERSITY

Real-space Multi-Grid method (RMG) Ø Ø Ø Ø Ø Density functional equations solved directly on the grid Multigrid techniques remove instabilities by working on one length scale at a time Non-periodic

2 Real-space Multi-Grid method (RMG) Ø Ø Ø Ø Ø Density functional equations solved directly on the grid Multigrid techniques remove instabilities by working on one length scale at a time Non-periodic boundary conditions are as easy as periodic Compact Mehrstellen discretization A f ] + B[( V + V ) f ] = e B[ Sf ] [ i eff NL i i i Allows for efficient massively parallel implementation Largest run used 139,392 CPU cores and 8,712 GPU's, > 6.5 PF RMG open source sourceforge.net/projects/rmgdft/ Quantum transport: later in 2017 Amyloid β atoms Multigrids Basis Cray XK7 ORNL Performance on 3,872 Cray XK7 (K20x GPU) Blue Water nodes: 1.14 PFLOPS 1 node = 16 Opteron cores+ 1 Nvidia K20x GPU

RMG Code Downloads v Downloads: https://www.rmgdft.org v Code & binaries (for Linux distr., Mac, Win) v Blue Waters portal for Cray XE6/XK7 https://bluewaters.ncsa.illinois.

3 RMG Code Downloads v Downloads: v Code & binaries (for Linux distr., Mac, Win) v Blue Waters portal for Cray XE6/XK7 v Part of NSF s Sustained Petascale Performance benchmarks v Graphical User Interface (GUI) for input & output v Examples, documentation & discussion forum on sourceforge ~ 2,000 downloads since Nov ~60 papers Your feedback is valuable!

4 Performance and scalability: RMG vs. plane waves 3000 atoms (1000 water molecules, Vanderbilt USPP, initial electronic quench)

RMG and GPU accelerators Programming models CPU vs GPU CPU - high clock speed,

Memory latency hidden by caches and out of order execution.

GPU lower clock speed, large number of weaker execution units.

5 RMG and GPU accelerators Programming models CPU vs GPU CPU - high clock speed, smaller number of powerful execution units. Memory latency hidden by caches and out of order execution. Good single threaded performance. GPU lower clock speed, large number of weaker execution units. Memory latency hidden by high thread counts. Poor single-threaded performance. Large fraction of RMG codebase is well adapted for the GPU model. Large data sets required though. CPUs work better for very small problems.

6 GPU Speed-up Bulk copper 256 atom cell initial electronic quench GPU speed-up This problem is too small for > 32 GPU nodes

7 GPU performance workstation level Workstation calculation Dual Xeon E5-2630v2 workstation. Total of 12 CPU cores and 32GBytes RAM. New Nvidia GP100 Pascal, 16GBytes, HBM memory and 5.3 TFLOPS double precision. Test problem 256 atom copper cell Vanderbilt ultrasoft pseudopotentials with 18 beta functions/atom. Total of 1536 electronic orbitals. PBE XC functional. Execution time dominated by eigensolver and large matrix operations CPU only run required 94.4 seconds/scf step. CPU/GPU run required 19.7 seconds/scf step. A single GPU produced a speedup by a factor of 4.8! (GP100 received in March 2017, code not optimized yet, further improvement expected)

8 GPU performance: supercomputer level Quantum computing candidate Nitrogen-vacancy complex in diamond: spin-polarized DFT atoms, orbitals. Norm-conserving pseudopotentials (FHI). Cray XE6 results using 5184 nodes CPU cores (32 cores/node) Initial electronic quench seconds, 52.8 GFlops/node Cray XK7 results using 3456 nodes CPU cores (16 cores/node), 3456 Nvidia GPU accelerators (1/node) Initial electronic quench seconds, GFlops/node Cray XE nodes 99.8 Gflops/node 30% of peak. v Part of NSF s Sustained Petaflops Performance benchmark

Introduction to DNA Replication v Klenow Fragment (KF) from DNA Polymerase I

templates Ø High replication rate, 100s of base pairs per second Ø Low error

current when the enzyme has just added a DNA base Ø Potential for DNA

open state: not synthesizing closed state: synthesizing T. J. Olsen, Y.

9 Introduction to DNA Replication v Klenow Fragment (KF) from DNA Polymerase I is an enzyme that synthesizes double-stranded DNA from single-stranded templates Ø High replication rate, 100s of base pairs per second Ø Low error frequencies, 1 error in 10 6 base pairs v Experiments observe a different current when the enzyme has just added a DNA base Ø Potential for DNA sequencing, if different bases lead to different currents. open state: not synthesizing closed state: synthesizing T. J. Olsen, Y. Choi, P. C. Sims, O. T. Gul, B. L. Corso, C. Dong, W. A. Brown, P. G. Collins, and G. A. Weiss, JACS 135, 7855 (2013)

Our previous results on nanotube sensing v v

between DNA bases Nucleotide substitutions can

Calculated signals can identify T and G, but

alpha-thiol datp dc + 6- chloro dgtp Cl x x S

Pugliese, O. T. Gul, Y. Choi, T. J. Olsen, P.

10 Our previous results on nanotube sensing v v Gate potential scanning helps to distinguish between DNA bases Nucleotide substitutions can be used to enhance the current difference, v Calculated signals can identify T and G, but cannot distinguish between C and A dt + alpha-thiol datp dc + 6- chloro dgtp Cl x x S 2-thio dttp + da 2-thio dctp + dg S S K. M. Pugliese, O. T. Gul, Y. Choi, T. J. Olsen, P. C. Sims, P. G. Collins, and G. A. Weiss, JACS 137, 9587 (2015)

11 Self-Consistent Electron Transport for >10k Atoms lead1 S L G L Conductor region R Coupling between lead and conductor S G R lead2 Coupling between lead and conductor v Non-equilibrium Green s function (NEGF) method in a basis of variationally optimized orbitals Lu, Meunier, Bernholc, Phys. Rev. Lett v Optimal basis set: Ø Variationally-optimized localized orbitals: a smaller and more complete basis set than in other localized-orbital methods. Fattebert & Bernholc, Phys. Rev. B 2000 v Green s functions are calculated iteratively with linear scaling. v Multi-level parallelization Ø MPI between nodes Ø Full multi-core and multi-gpu support Ø Scales to thousands of nodes We can calculate conductances and I-V curves for multi-terminal devices from first principles (DFT) for large systems (>10,000 atoms) I(V ) = 2e2 h T(E,V )[ f (E µ L ) F(E µ R )]de

12 Long Si nanowire with Al 2 O 3 coating Ø Nanowire (NW) size: Si diameter: 5 nm, Length 5.64 nm Al 2 O 3 thickness: 1nm Ø A transport gap of 0.5 ev appears for this nanowire. Ø Transmission is nearly zero at the Fermi level, no tunneling current. Ø Currents are calculated with source-drain bias of 100 mev. Open Closed

13 Si nanowire with Al 2 O 3 coating: sensitivity Sensitivity = I "#$% I '(")$* I "#$% + I '(")$* Ø Short Si nanowire (2.8 nm): At effective gate of 0.4 V, the sensitivity is about 10%, the current difference between open and closed states is 0.18 µa. At negative gate, the surface states dominate: not suitable for sensing. Ø Long Si nanowire (5.6 nm): At effective gate of V or 0.35 V, the sensitivity is 8% and the current difference is 0.02 µa In subthreshold region, the sensitivity is 10%, but the current is very small (0.3 na)

14 Si Slab with Al 2 O 3 Passivation Si slab thickness: 1 nm, length: 6.7 nm, Al 2 O 3 thickness: 1 nm Open Closed Width: 3.8 nm Width: 7.6 nm Ø Transport gap of 0.6 ev for both widths Ø Subthreshold (-0.2 to 0.4 V): sensitivities are large but currents are small. Ø Turn-on state (gate at -0.3 and 0.5 V): Sensitivity is reduced when slab width increases, because the base current becomes larger. The absolute current difference does not change dramatically.

Nanoribbon Confinement and Transmission Length Width v Leads: bulk graphene v Conductor part: armchair nanoribbon width from 3 to 15 atomic layers length from 5 to 14 atomic layers v Dangling bonds

15 Nanoribbon Confinement and Transmission Length Width v Leads: bulk graphene v Conductor part: armchair nanoribbon width from 3 to 15 atomic layers length from 5 to 14 atomic layers v Dangling bonds are saturated with hydrogens v Number of atoms in NEGF calculations 800 to 1300 atoms depending on width/length L = 14, W = 12 atomic layers n= n=1 2 3 v Finite-size GNRs confined between contacts lead to Fabry-Perot interference patterns Liang, Park et al, Nature 2001; Rickhaus, Maurand et al, Nat. Com. 2013; Yannouleas, Romanovsky, Landman, JPC 2015, Sci. Rep v The interference patterns depend on length, width, GNR index and passivation. v Interfaces can also induce localized states.

Nanoribbon Families and Suitability for Transistors Width = 12, mod(n,3) = 0 Band gap is large with no localized states inside band gap for a finite length

Width = 13, mod(n,3) = 1 Pure nanoribbon has a large band gap, while there are localized states inside the band gap for a finite length nanoribbon.

bandgap family It is hard to have a small current for OFF state It is not a good candidate for a transistor, which requires high ON/OFF ratio.

16 Nanoribbon Families and Suitability for Transistors Width = 12, mod(n,3) = 0 Band gap is large with no localized states inside band gap for a finite length nanoribbon. OFF state current can be very small because of the big band gap It is a good candidate for a transistor, which requires high ON/OFF ratio. Width = 13, mod(n,3) = 1 Pure nanoribbon has a large band gap, while there are localized states inside the band gap for a finite length nanoribbon. Transmission peaks inside the gap decrease dramatically with length OFF-state current can be very small because of the big band gap Width = 14, mod(n,3) = 2 Small bandgap family It is hard to have a small current for OFF state It is not a good candidate for a transistor, which requires high ON/OFF ratio. For a moderate-size device, nanoribbons with mod(n,3) = 0 are the best choice. The first transmission peak appears at a small energy for a long ribbon. The nanoribbon needs to be long enough to avoid direct tunneling current.

17 Nanoribbon Transistor Simulation Best nanoribbon for transistor: mod(n,3) = 0 family of armchair nanoribbons Nanoribbon: 18 atomic-layers wide 40 atomic-layers long Graphene sheet as source and drain 2 layers of BN as the insulator A few layers of Al as the gate 4536 atoms in the NEGF calculation Source-drain bias: 50 mv At a positive gate of 0.4 V, the device is turned on with ON/OFF ratio = 500 The small ON/OFF ratio in part due to too small band gap in calculations (DFT artefact).

18 Summary q RMG -- a petaflops-capable open source electronic structure code v Scales to 20k nodes and 200k CPU cores v Effective use of GPUs and multi-core CPUs v Pseudopotential libraries: ultrasoft & norm-conserving v Cray installation files, Linux, Windows and Mac binaries v Graphical user interface (GUI) v Released under GPL: v Blue Waters community Portal: v Part of NSF s Sustained Petascale Performance benchmarks q Applications v Monitoring of DNA replication using nanotubes and Si nanowires v Potential for DNA sequencing v Nanoscale devices for Post Moore s Law era v Nanoribbon-based transistors are feasible Ø Quantum interference patterns in small structures Ø Specific nanoribbon indices are required

CP2K PERFORMANCE FROM CRAY XT3 TO XC30. Iain Bethune Fiona Reid Alfio Lazzaro

CP2K PERFORMANCE FROM CRAY XT3 TO XC30 Iain Bethune (ibethune@epcc.ed.ac.uk) Fiona Reid Alfio Lazzaro Outline CP2K Overview Features Parallel Algorithms Cray HPC Systems Trends Water Benchmarks 2005 2013