The Next-Generation Supercomputer Project and the Future of High End Computing in Japan

Transcription

1 10 May 2010 DEISA-PRACE Symposium The Next-Generation Supercomputer Project and the Future of High End Computing in Japan To start with Akira Ukawa University of Tsukuba Japan Status of the Japanese Next-Generation Supercomputer Project Forming the Japanese HPC Infrastructure Toward exascale computing Prepared with Mitsuhisa Sato, Univ. of Tsukuba 1

2 What I wish to do today report on the status of the Japanese 10 Petaflops supercomputer project - HPC technology discuss recent effort toward building up the Japanese HPC infrastructure - HPC sociology touch upon issues toward exaflops - HPC philosophy 2

3 Status of the Next Generation Supercomputer (NGS) Project 3

4 Background: our government plan 3 rd Science and Technology Basic Plan FY Next-generation supercomputing technology was selected as one of the key technologies of national importance Development of the Next-Generation Supercomputer Cf. other key technologies: Free electron laser, ocean and earth observation and exploration, rapid breeder reactor, space technologies 4 th Science and Technology Basic Plan FY (now under discussion) Exaflops class HPC Technology New chip device, software, hardware 4

5 Next-Generation Supercomputer Project Goals: 1. Development of 10 Pflops-class system 2. Development of grand challenge applications in nano science and life science 3. Federation of the 10 Pflops-class and supercomputers installations throughout Japan 4. Buildup of a research center in computational science around the 10 Pflops-class system Project period Japanese FY 2006 to 2012 Project budget B Institution responsible for the computer development RIKEN 5

6 Original schedule of the Project Big jolt in November 2009 JFY2006 JFY2007 JFY2008 JFY2009 JFY2010 JFY2011 JFY2012 System R&D Conceptual design Detailed design Prototyping/evaluation/production Tuning Grand Challenge Applications Nano applications Life applications R&D and evaluation R&D and evaluation verification verification Building Computer building Research building Design Design Construction Construction 6

7 Political turmoil in Japan last November 31 August 2009 Landslide victory by People s Democratic Party (first real change of power since 1951) 18 September: Government Revitalization Unit (GRU) is set up; starts reexamination of FY2010 budget 13 November: GRU Working Group, after an 1-hour public hearing, recommends freezing of the Supercomputer Project ; many science & technology budget also recommended cut. Late November-early December: appeals by many academic communities against the GRU s reccomendation 16 December: Government decides to proceed with the Project 7

8 NGS (Next Generation Supercomputer) System Configuration 8

9 System overview Compute nodes Multi-dimensional mesh/torus network Control server Configs control Admin server Control/admin network Local file system Global IO system Frontend server internet jobs Users files Global file system 9

10 Compute Nodes and network #compute nodes (CPUs): > 80,000 #cores: > 640,000 Peak performance: > 10PFLOPS Memory: > 1PB (16GB/node) 5GB/s (peak) x 2 Logical 3-dimensional torus network Peak bandwidth: 5GB/s x 2 for each direction of logical 3-dimensional torus network bi-section bandwidth: > 30TB/s SPARC64 TM VIIIfx 5GB/s(peak) x 2 Compute Node CPU: 128GFLOPS (8 Core) Core Core Core Core SIMD(4FMA) SIMD(4FMA) Core Core SIMD(4FMA) SIMD(4FMA) Core Core SIMD(4FMA) 16GFlops SIMD(4FMA) SIMD(4FMA) 16GFlops 16GFlops 16GFlops 2FMAx2SIMD 16GFlops 16GFlops 16GFLOPS 16GFlops L2$: 5MB 5GB/s(peak) x 2 z y x 64GB/ MEM: 16GB s 5GB/s (peak) x 2 Logical 3-dimensional torus network for programming Courtesy of FUJITSU Ltd. 10

11 CPU Features (Fujitsu SPARC64 TM VIIIfx) 8 cores 2 SIMD operation circuit 2 Multiply&Add floating-point operations (SP or DP) are executed in one SIMD instruction 256 FP registers (double precision) Shared 5MB L2 Cache (10way) Hardware barrier Prefetch instruction Software controllable cache - Sectored cache Performance 16GFLOPS/core, 128GFLOPS/CPU Reference: SPARC64 TM VIIIfx Extensions 45nm CMOS process, 2GHz 22.7mm x 22.6mm 16GF/core(2*4*2G) 760 M transisters 58Wat 30 by water cooling) 11

12 System board and rack 4 CPU and 4 ICC (network chip)/board 467x554mm 24 boards (96 CPU s)/rack 796mm 750mm 2060mm System Board 4 CPU and Network Interface/board CPU ICC Photo courtesy FUJITSU Limited 12

13 System environment OS: Linux-based OS on compute nodes Frontend server Compute node File system: 2-level (local/global) system Permanent user files on the global system directly accessible via NSF-like system from frontend server Staged in/out to local system for job execution User terminal Remote login Login, file handling, Compile IDE Compiler Debugger Profiler access Global file system Job control debugging Job control node staging Parallel job debugger Parallel job Parallel job I/O Local file system 13

14 Programming features Hybrid programming model with multi-threads and MPI Fortran 2003XPFortran C C++ Optimized compiler (SIMD/256 FP registers/sector cache) MPI based on MPI-2.1 specification Numerical libraries (BLAS, LAPACK, FFTW, SSL-II (Fujitsu Library) Debugger and Profiler Optimizations and SIMD operation generations by compilers in a core multi-threads by OpenMP directives and/or automatic parallelization by compiler on a CPU Parallelization by MPI libraries or programmming with a high-level Fortran language XPFortran among CPUs CPU CPU CPU CPU core core core core core core core core Memory Memory Memory Memory 14

15 Cite for the Next Generation Supercomputer Kobe Kobe Tokyo 450km (280miles) west from Tokyo Building cite Kobe airport 15

16 Photo of building (as of January, 2010) Research Building Computer building Chillers Substation Supply Research building Computer building office office Computer room NGS office office Air conditioner room Air cooling system office office Machine room office Machine room Computer room Global file system Air cooling system 16

17 Inside the building Computer room 3F Making a double floor Chillers Solar panels on the top Research building 17

18 Schedule of the project We are here. FY2006 FY2007 FY2008 FY2009 FY2010 FY2011 FY2012 System Conceptual design Detailed design Prototype, evaluation Production, installation, and adjustment Tuning and improvement Buildings Applications Next-Generation Integrated Nanoscience Simulation Next-Generation Integrated Life Simulation Computer building Research building Development, production, and evaluation Development, production, and evaluation Design Design Construction Construction Verification Open to the project Verification 18

19 Forming the Japanese HPC Infrastructure 19

20 Three topics to be covered New Institute for Computational Science in Kobe Strategic Field Program for NGS - Strategic Usage of the Next-Generation Supercomputer - Formation of national HPCI (High Performance Computing Infrastructure) and Consortium 20

21 Next-Generation Supercomputer Project Goals: 1. Development of 10 Pflops-class system 2. Development of grand challenge applications in nano science and life science 3. Federation of the 10 Pflops-class and supercomputers installations throughout Japan 4. Buildup of a research center in computational science around the 10 Pflops-class system Project period Japanese FY 2006 to 2012 Project budget B Institution responsible for the machine development RIKEN embryonic to HPCI/Consotrium New Institute in Kobe 21

22 New Institute in Kobe Advanced Institute for Computational Science Objectives Manage the Next-Generation Supercomputer (NGS) Carry out computational science research Lead HPC in Japan Starting date: June 2010 research group to start in October 2010 Emphasis Strong collaboration between computational and computer scientists Research of future HPC systems after NGS Fostering young scientists with expertise in both computational and computer science 22

23 Planned organization of the Institute Director Advisory Board Search Committee Strategic planning Division strategic planning independent of administration Buildup of national research systen for HPC, coordination of HPC-related research organizations and univsertities in Japan, International federation and activities Computational Science Division advanced methodologies Quantum systems Group Particles and nuclei, reaction-catalysis dynamics, functional materials Particle systems Group Structure formation in Universe, Protein dynamics, Virus functions Continuous systems Group Blood flow, Global climate modeling, Mega earth quake Complex systems Group Batteries, Disaster simulation, Nuclear fusion, Cell simulation Discrete systems Group Bioinformatics, Socio-economic phenomena, Pandemics Computer Science Division Systems software Group Large-scale I/O, communications software development Parallel software Group Parallel programming languages, optimized compiler technologies Applications software Group Parallel algorithms, middleware development Architecture Group Future parallel architecture based on applications needs Operation Division Operations Group NGS operation, hardware maintenance, software maintenance Systems research Group system optimization, application benchmarking User support Group user code analyses, code optimization support 7 23

24 Strategic Field Program for NGS In order to put NGS to strategic use, Government selected 5 strategic fields in science and technology for importance from national view point For each field, Government also selected a core institute. Each core institute is responsible for organizing research and supercomputer resources in the respective field and its community, for which they receive priority allocation of NGS resources funding to achieve the research goals 24

25 Supercomputer installation in Japan Computer Centers of major universities Academic research institutions/organizations Each major field has its own supercomputer facility e.g., High energy physics Condensed matter physics Astrophysics Genetics/bioinformatics Government labs AIST/JAEA/NIMS. Earth Simulator belongs to JAMSTEC (Japan Marine science and Technology Center) 25

26 Strategic Fields and core institutes strategic field Life Science & Medicine New materials & Energy Global change prediction Next generation Engineering Matter & Universe proteins genome body nano materials new device Novel battery tissue,organ cells Multilayers of life core institute RIKEN Institute for Solid State Physics U. Tokyo Earth Simulator Center JAMSTEC Institute for Industrial Science U. Tokyo Center for Comp. Science U. Tsukuba Life science Community Supercomputer resources materials science Community Supercomputer resources Earth science Community Supercomputer resources Engineering Community Insdutry Supercomputer resources Basic science Community Supercomputer resources 26

27 Next 10 years Next-Generation supercomputer R&D operation Advanced Institute for Computational Science in Kobe Strategic Field Program Feasibility study Prepara tion Full operation of the program Consortium/HPCI (HPC Infrastructure) Prepara tion Full operation R&D toward Exascale systems? 27

28 High-performance Computing Infrastructure (HPCI) and (Japanese HPC) Consortium Background: The goal of the NGS has been reconsidered by the new government. More emphasis on user point of view than the developer s view. Redesign of the Project HPCI: High-Performance Computing Infrastructure Integrated operation of NGS with other institutional supercomputers Seamless access from institutional supercomputers and user's machines to NGS. Large-scale storage systems shared by NGS and others. (Japanese HPC) Consortium To play the role of main body to run HPCI (and design HPCI). To organize the computational science communities as whole 28

29 (Japanese HPC) Consortium A Uinv center D Univ center University/Institutional Supercomputer Centers C Univ. center B Univ Center B Institute center A Institute center Industry Steering Committee NII Kobe Institute User community research institutes life environment Basic science Materials energy manifacturing A field core institute Industry use A univ supercomputer C uinv supercomputer D univ supercompute supercomputer A institute supercomputer Network and storage Industry supercomputer network NGS HPCI network A field supercomputer D field supercomputer B field supercomputer E field supercomputer C field supercomputer 29

30 Present status and pending issues Status The basic plan for the Consortium/HPCI formulated and made public for public comments Government will decide the final plan in late May, and call for participation to the Consortium/HPCI The Consortium will start in June The biggest pending issue: How to share supercomputer resources on HPCI How to allocate supercomputer time, including those of NGS, on scientific basis 30

31 Toward exascale computing 31

32 Already lots of discussions led more by computer scientists than by computational scientists Science/technology drivers Climate, Environment, Energy, Materials, Basic science, etc Hardware issues e.g., DoD report Exascale Computing Study P. Kogge et al (2008) Software issues e.g., IESP Workshop series Santa Fe/Paris/Tsukuba/Oxfoed/Maui Co-design approach 32

33 Co-design and lattice QCD co-design integrates algorithm, software, hardware and data management design and development i.e., science needs and engineering feasibility are matched step by step in the computer development QCD (Quantum Chromodynamics) in high energy physics is a historic example where this approach has had tremendous successes: EU APE series, QPACE, Japan PACS series USA QCDOC, QCDSF, 33

34 Quantum Chromodynamics Gross-Wilczek-Politzer 1973 Quantum field theory of quarks and gluons q f A L QCD O x x Knowing 1 Tr 8 1 coupling constant and 6 quark masses s Quark field Gluon field F F q f ia 1 Z A,, dadqdqo A, q, qe f s 4 m, m, m u defined over 4-dim space time will allow full understanding of hadrons and their strong interactions g s d 2 d 4 xl QCD s, m m c f q, m b f, m t QCD lagrangian Hadons(proton, neutron,pion etc) are composites of 3 or 2 quarks u u d proton=uud fulfilling Yukawa s dream of 1934 in a refined way 34

35 QCD as computation Wilson 1974 QCD is a local field theory; only nearest neighbor interaction Intensive computation and communication/node (for physical reasons) PU1 PU3 PU2 PU4 An ideal case for massively parallel architecture Spurred the development of (QCD-dedicated) massively parallel computers since 1980 s 35

36 Impact of lattice QCD machines on the supercomputer development BlueGene/L,P QCDOC(USA) GF11(USA) PACS-CS(JPN QCDPAXJPN QCDSP(USA APE100Italy CP-PACSJPN : Top500 No. 1 (1996) 36

37 QCD has been a vertical co-design computer science Hardware (CPU, memory, network, I/O) Software(OS, compiler, langauge ) Novel technologies particle physics Modelling, algorithm, data management 37

38 Challenges over the next decade will require amalgamation of multiple of science fields An example from basic science: The Origin of Heavy Elements single hadron r < [cm] Nuclei made of multiple hadrons r [cm] Supernova explosions and heavy element synthesis particle physics nuclear physics astrophysics Amount of computing 0.11 PFlops PFlops 1 Exaflops? 38

39 Many of exaflops challenges will require horizontal as well as vertical co-design computer science Hardware (CPU, memory, network, I/O) Software(OS, compiler, langauge ) Novel technologies Science Field A Science Field B Science Field C multiscale modelling, multiscale algorithm, data management 39

40 Expected funding and plans in Japan JST CREST (Core Research for Evolutional Science and Technology) for "new HPC technologies" is being proposed M JPY(5M USD) for 5-7 years (4-5 B JPY in total) To be software-oriented, rather than hardware. Research for systems after NGS to be conducted at the new Kobe Institute 4 th Science and Technology Basic Plan FY (now under discussion) Exaflops class HPC Technology New chip device, software, hardware 40

41 Conclusions System development of NGS is proceeding as planned. A prototype has been built, and production is starting up. Call for a nickname for NGS announced Serious work underway to organize the HPC communities and institutions in Japan for efficient usage of NGS and healthy development of HPC. Many challenges toward exascale, clearly calling for a coordinated endeavor by the international HPC community. 41