PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich Event, Location, Date

Size: px

Start display at page:

Download "PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich Event, Location, Date"

Berenice Booker
5 years ago
Views:

1 PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich Event, Location, Date

2 Outline What is PRACE Where we stand What comes next Questions 2

3 Outline What is PRACE Where of we stand What comes next Questions 3

4 Supercomputing Drives Science through Simulation Environment Weather/ Climatology Pollution / Ozone Hole Ageing Society Medicine Biology Materials/ Inf. Tech Spintronics Nano-science Energy Plasma Physics Fuel Cells 4

the Scientific Case HPCEUR HET Signature of the MoU Submission of an FP7 project

5 History and First Steps Production of the HPC part of the ESFRI Roadmap; Creation of a vision, involving 15 European countries Bringing scientists together Creation of the Scientific Case HPCEUR HET Signature of the MoU Submission of an FP7 project proposal Approval of the project Project start PRACE Initiative

HET: The Scientific Case Weather, Climatology, Earth

variations weather and flood events Astrophysics,

structures which span a large range of different

QCD, ITER Material Science, Chemistry, Nanoscience

nanoscience the determination of electronic and

chromatin dynamics, large scale protein dynamics,

systems, medicine Engineering complex helicopter

6 HET: The Scientific Case Weather, Climatology, Earth Science degree of warming, scenarios for our future climate. understand and predict ocean properties and variations weather and flood events Astrophysics, Elementary particle physics, Plasma physics systems, structures which span a large range of different length and time scales quantum field theories like QCD, ITER Material Science, Chemistry, Nanoscience understanding complex materials, complex chemistry, nanoscience the determination of electronic and transport properties Life Science system biology, chromatin dynamics, large scale protein dynamics, protein association and aggregation, supramolecular systems, medicine Engineering complex helicopter simulation, biomedical flows, gas turbines and internal combustion engines, forest fires, green aircraft, virtual power plant 6

First success: HPC in ESFRI Roadmap The European Roadmap for Research Infrastructures is the first comprehensive definition at the European level Research Infrastructures are one of

7 First success: HPC in ESFRI Roadmap The European Roadmap for Research Infrastructures is the first comprehensive definition at the European level Research Infrastructures are one of the crucial pillars of the European Research Area A European HPC service impact foreseen: strategic competitiveness attractiveness for researchers supporting industrial development 7

8 Second success: The PRACE Initiative Memorandum of Understanding signed by 15 States in Berlin, on April 16, 2007 France, Germany, Spain, The Netherlands, UK committed funding for a European HPC Research Infrastructure (LoS) New: 8

from 14 countries Project duration: January 2008 December 2009

9 Third success: The PRACE Project EU approved the PRACE Preparatory Phase Project (Grant: INFSO-RI ) 16 Partners from 14 countries Project duration: January 2008 December 2009 Project budget: 20 M, EC funding: 10 M Kickoff: Jülich, January 29-30,

10 PRACE Objectives in a Nutshell Provide world-class systems for world-class science Create a single European entity Deploy 3 5 systems of the highest performance level (tier-0) Ensure diversity of architectures Provide support and training PRACE will be created to stay 10

11 Outline What is PRACE Where we stand What comes next Questions 11

12 After the First Successful Year: Project review March 5-6, 2009, in Brussels A full day of presentation of the project results of year1 Coordinator and all work packages presented PRACE is the collaborative achievement of over 250 persons at the 16 partner sites Expertise includes a Chairman of the Board, (coordinator), executives, scientists, programmers, legal experts, The project made very good progress in many areas 12

13 PRACE Work Packages WP1 Management WP2 Organizational concept WP3 Dissemination, outreach and training WP4 Distributed computing WP5 Deployment of prototype systems WP6 Software enabling for prototype systems WP7 Petaflop/s systems for 2009/2010 WP8 Future petaflop/s technologies 13

14 Selected Results and Highlights of 2008 Applications Systems/Architectures Training and Outreach 14

15 Software Enabling for Petaflop/s Systems Prototype Systems Representative applications Categorisation of Applications Benchmarks Benchmarks Application Requ. Petascaled applications Optimised applications Analysis Architectures Petascaling Optimisation Libraries&Models 15

Categorisation of Applications Benchmark applications should be representative of European HPC usage We conducted surveys of PRACE partners HPC systems and major applications Collecting various

16 Categorisation of Applications Benchmark applications should be representative of European HPC usage We conducted surveys of PRACE partners HPC systems and major applications Collecting various interesting data for 24 systems and 69 applications Quantitative basis for selecting representative applications Disseminated as Technical Report Plasma Physics 3.3 Computational Engineering 3.7 Life Sciences 5.3 Astronomy & Cosmology 5.8 Earth & Climate 7.8 CFD 8.6 Other 5.8 Particle Physics 23.5 Computational Chemistry 22.1 Condensed Matter Physics

17 Application Requirements Analysis of representative applications Ported to relevant architectures Result: Quantitative data from initial porting Supplemented by user survey Sent to Top 10 users in each PRACE country Questions covered The user Usage patterns HPC infrastructure Upcoming algorithms Analysed almost 70 responses from these major users Machine Fraction [%] Job Requirements Size Length ,1 Minimal execution time [h] 17

18 Representative Benchmark Suite Defined a set of applications benchmarks To be used in the procurement process for Petaflop/s systems 12 core applications, plus 8 additional applications Core: NAMD, VASP, QCD, CPMD, GADGET, Code_Saturne, TORB, ECHAM5, NEMO, CP2K, GROMACS, N3D Additional: AVBP, HELIUM, TRIPOLI_4, PEPC, GPAW, ALYA, SIESTA, BSIT Each application will be ported to appropriate subset of prototypes Synthetic benchmarks for architecture evaluation Computation, mixed-mode, IO, bandwidth, OS, communication Applications and Synthetic benchmarks integrated into JuBE Juelich Benchmark Environment 18

Mapping Applications to Architectures Identified affinities and priorities Based on the application analysis - expressed in a condensed, qualitative way Need

19 Mapping Applications to Architectures Identified affinities and priorities Based on the application analysis - expressed in a condensed, qualitative way Need for different general purpose systems There are promising emerging architectures Will be more quantitative after benchmark runs on prototypes E = estimated 19

20 Selected set of Prototypes Process Call evaluation selection Approval by EC Funding 2.2 M (50% of cost in average) Site FZJ Germany CSC-CSCS Finland+Switzerland CEA-FZJ France+Germany NCF Netherlands BSC Spain HLRS Germany Architecture Vendor/Technology MPP IBM BlueGene/P MPP Cray XT5/XTn - AMD Opteron SMP-TN Bull et al. Intel Xeon Nehalem SMP-FN IBM Power 6 Hybrid fine grain IBM Cell + Power6 Hybrid coarse grain NEC Vector SX/9 + x86 M7.1 July 16: EC review 20

21 Installed prototypes IBM BlueGene/P (FZJ) IBM Power6 (SARA) Cray XT5 (CSC) Intel Nehalem/Xeon (CEA/FZJ): expected installation date 4/ IBM Cell/Power (BSC) NEC SX9, vector part (HLRS)

22 Summary of current prototype status milestone IBM BlueGene/P at FZJ IBM Power6 at SARA Cray XT at CSC IBM Cell/Power at BSC NEC SX9/x86 at HLRS Intel Nehalem/Xeon at CEA/FZJ system installed yes yes yes yes partly (vector) no system in production yes yes yes yes partly (vector) no technical assessment yes nearly yes started no no evaluation of communication and I/O infrastructure evaluation and benchmarking of user applications yes nearly yes started no no started started started started started no 22

Web site and the dissemination channels The PRACE web presence with news, events, RSS feeds etc. http://www.prace-project.

23 Web site and the dissemination channels The PRACE web presence with news, events, RSS feeds etc. Alpha-Galileo service: 6500 journalists around the globe: Belief Digital Library HPC-magazines PRACE partner sites, top 10 HPC users The PRACE website, 23

flyers, posters, badges, t-shirts, USB-keys, badges etc.

24 DE1 PRACE Dissemination Package PRACE WP3 has created a dissemination package including templates, brochures, flyers, posters, badges, t-shirts, USB-keys, badges etc. The PRACE logo PRACE USB-key Heavy Computing 10^15: the PRACE t-shirt 24

25 Slide 24 DE1 The slides are very good and lively However, you should bring some structure into the presentation. I propose to start with a slide on WP3 objectives - Dissemination... - Industry Collaboration... - Training... Highlight explicitely at the appropriate places that the Milestones were successfully reached. (Don't expect that the reviewers read everything) Dietmar Erwin;

26 PRACE booth at ISC, ICT, SC PRACE had an exhibition booth at ISC 08 Dresden, D SC08 Austin, Texas, US ICT 2008, Lyon, F PRACE booth at SC08 PRACE booth at ICT

27 Selected Events Industry Seminar PRACE Award ISC 08 Best student paper on petascaling to Dominik Göddeke, Dortmund 26

28 Industrial Competitiveness: Europe goes HPC / Attendeed companies 27

Training: Survey of HPC education and training needs The Top 10 users at each participating PRACE member site were invited to participate in completing the survey.

29 Training: Survey of HPC education and training needs The Top 10 users at each participating PRACE member site were invited to participate in completing the survey. The data was obtained from the most comprehensive evaluation of user training requirements Over 90% of respondents believed they would benefit from formal training in the following areas: performance optimization, debugging tools and techniques, code testing and compiler optimisations; Over 90% of users considered that there is an important need for improved HPC training programmes Request to use survey from Hong Kong and US 28

Training: summer School PRACE Petascale Summer School, August 26-29, Stockholm, Sweden; It attracted 31 students, representing all PRACE member countries plus South Africa.

30 Training: summer School PRACE Petascale Summer School, August 26-29, Stockholm, Sweden; It attracted 31 students, representing all PRACE member countries plus South Africa. In the anonymous feedback answered by 26 students, the School received excellent grades; Training material published on the PRACE website. Special access to large European systems - BG/P,Cray XT5, IBM Power6, and CELL - was offered to the students in conjunction with lectures about programming models, hardware, development tools and optimisation/debugging PRACE Summer School took place at PDC, Stockholm 29

2009 training: Winter School PRACE Winter School, February 9-13, 2009, Athens, Greece; 48 registered attendees (78 applications); Training material

31 2009 training: Winter School PRACE Winter School, February 9-13, 2009, Athens, Greece; 48 registered attendees (78 applications); Training material is published on the PRACE web site. The PRACE Winter School at the OTE academy, Athens HPC training and HPC training events on the PRACE website 30

32 Outline What is PRACE Where we stand What comes next Questions 31

33 PRACE All-Hands meeting February

34 The next important challenges Get the initial commitments by Government converted into signed contracts Governance document Report on Operating model Doc on funding agreement selection of next production systems Draft contract Peer review process PRACE CONTRACT Month 15 Month 18 Month 24 33

35 How to get involved? If you are a national coordinator of HPC activities and your country is not yet a member: Join the PRACE Initiative! Port your code to the PRACE Prototypes Prototypes will mainly be used project-internally, but Prototypes will also be made available to selected users for testing/porting purposes using a light-weight peer-review process See: Participate in the PRACE training events 34

36 Accessing the future PRACE RI Disclaimer: this is work in progress, not yet confirmed Access Model Based on peer-review: the best systems for the best science Three types of resource allocations Test / evaluation access Project access for a specific project, grant period ~ 1 year Programme access resources managed by a community Free-of-charge Funding Mainly national funding through partner countries European contribution Access model has to respect national interests (ROI) 35

37 Prototypes: Change Status to Green milestone IBM BlueGene/P at FZJ IBM Power6 at SARA Cray XT at CSC IBM Cell/Power at BSC NEC SX9/x86 at HLRS Intel Nehalem/Xeon at CEA/FZJ system installed yes yes yes yes partly (vector) no system in production yes yes yes yes partly (vector) no technical assessment yes nearly yes started no no evaluation of communication and I/O infrastructure evaluation and benchmarking of user applications yes nearly yes started no no started started started started started no 36

38 Prototypes: Change Status to Green milestone IBM BlueGene/P at FZJ IBM Power6 at SARA Cray XT at CSC IBM Cell/Power at BSC NEC SX9/x86 at HLRS Intel Nehalem/Xeon at CEA/FZJ system installed yes yes yes yes yes yes system in production yes yes yes yes yes yes technical assessment yes yes yes yes yes yes evaluation of communication and I/O infrastructure evaluation and benchmarking of user applications yes yes yes yes yes yes yes yes yes yes yes yes 37

39 Refine Costs Analysis / Estimates Initial: simplified scenarios, using market survey, vendor input and partners experience Updated and refined iteratively Consistent with the a priori PRACE initiative estimates in

40 Market watch Top 500 watch (Petascale worldwide ) Top 10 Top 5 Late Pflops 1 Pflops Late Pflops 2 Pflops Late Pflops 5 Pflops 39

41 Applications: Petascaling and Optimisation Petascaling Mixed-mode parallelisation Load balancing Minimisation of communication overheads Parallel I/O Checkpointing Optimisation Optimising serial performance CPU Memory Optimising both for general-purpose architectures and specialised architectures Algorithmic optimisations PRACE will disseminate best practice in these areas 40

42 Libraries and Programming Models Classification of Benchmark applications Current programming models MPI, OpenMP, mixed-mode, PGAS and other future programming models Accelerator Languages CUDA, RapidMind, opencl Petascale libraries 41

Future PRACE Events Second scientific seminar in 11.-13.

43 Future PRACE Events Second scientific seminar in May 2009 in Amsterdam in collaboration with DEISA2 (DEISA PRACE Symposium) The Second industry seminar Toulouse, 7-8th September organised by GENCI & GAUSS: covers also small and medium size enterprises 2nd PRACE Award at ISC09, Hamburg Exhibition booths at major events: ISC09 (June), SC09 (November) Five code porting and optimization workshops Finland, Sweden (2), Poland, Switzerland Additional training on GPU programming April, France 42

44 Integration into the HPC ecosystem Software builds on the developments in the DEISA project Synergies through overlapping partnership between PRACE and DEISA Enhancements and adaption where needed 43

45 Procurement strategy Analysis of European procurement procedures completed Open Restricted Competitive dialogue Negociated Pre-commercial Work in progress Definition of general procurement process Definition of selection and evaluation criteria Evaluation process for offers by vendors 44

Potential sites and installation constraints Survey of PRACE partners Result: The spectrum of the building infrastructure of the PRACE partners can cover the

46 Potential sites and installation constraints Survey of PRACE partners Result: The spectrum of the building infrastructure of the PRACE partners can cover the requirements expected for future petascale systems and most site can evolve to fulfil the needs of future generations of systems. To be updated at intervals 45

47 Future Petaflop/s Computer Technologies beyond 2010 Assessment and evaluation of emerging multi-petascaletechnology following the requirements of HPC users Implementation of a strategy that guarantees a continuous HPC technology evaluation and system evolution within the PRACE Research Infrastructure Fostering the development of components for future multipetascale production systems in cooperation with European and international HPC industry PRACE EC Review March Brussels 46

48 Creation of STRATOS STRATOS - PRACE advisory group for Strategic Technologies Design and implementation of a permanent HPC research and technology platform Agreements of STRATOS with cooperation partners, i.e. Consortia for HPC development and research institutions (STRATOS contract) Definition of long-term work plan ( ) PRACE EC Review WP8 March 5-6, Brussels 47

49 Multi-Petaflop/s technology Assessment of technologies needed to develop future multi-petascale systems Multi- and many-core processors Computational accelerators High speed interconnects, storage architectures and I/O technologies Novel compilers and performance tools System management software Analysis of user requirements and selection of promising architecture options 48

50 Multi-Petaflop/s components and architectures Assessment and evaluation/development of very highly multi-threaded processors (e.g. tens of cores per processor) Novel languages like PGAS languages Novel high speed interconnects Novel memory and storage technologies Novel parallel file system solutions 49

51 Promising Architectures Selected System Architectures Thin node systems (e.g., sgi ICE & Sun constellation) Fat node systems (e.g., sgi UV) All types of hybrid system architectures with one exception MarieCell (since it was already selected as WP7 prototype) Components for multi-petaflop/s Architectures Novel multi- and many-core processors Compute accelerators Novel high speed interconnects Novel storage solutions and file systems with end to end data integrity features PGAS and streaming processing languages 50

52 Prototypes selected Sites Hardware/Software Porting effort CEA GPU/CAPS CINES-LRZ LRB/CS CSCS UPC/CAF EPCC FPGA 1U Tesla Server T1070 (CUDA, CAPS, DDT) Intel Harpertown nodes Hybrid SGI ICE2/UV/Nehalem-EP & Nehalem-EX/ClearSpeed/ Larrabee Prototype PGAS language compilers (CAF + UPC for Cray XT systems) Maxwell FPGA prototype (VHDL support & consultancy + software licenses (e.g., Mitrion-C)) Evaluate GPU accelerators and GPGPU programming models and middleware. (e.g., pollutant migration code (ray tracing algorithm) to CUDA and HMPP) Gadget,SPECFEM3D_GLOBE, RaXml, Rinf, RandomAccess, ApexMap, Intel MPI BM The applications chosen for this analysis will include some of those already selected as benchmark codes We wish to port several of the PRACE benchmark codes to the system. The codes will be chosen based on their suitability for execution on such a system. 51

53 Prototypes selected (cont d) Sites Hardware/Software Porting effort FZJ (BSC) Cell & FPGA interconnect LRZ RapidMind NCF ClearSpeed eqpace (PowerXCell cluster with special network processor) RapidMind (Streaming Processing Programming Paradym) X86, GPGPU, Cell ClearSpeed CATS 700 units CINECA I/O Subsystem (SSD, Lustre, pnfs) - March 10 th : Selection of prototypes targeting energy efficiency Extend FPGA-based interconnect beyond QCD applications. ApexMap, Multigrid, FZJ (QCD), CINECA (linear algebra kernels involved in solvers for ordinary differential equations), SNIC Astronomical many-body simulation, Iterative sparse solvers with preconditioning, finite element code, cryomicrotome image analysis - 52

54 Thank you? Special thanks to all project collaborators for the content of the presentation 53

55 Backup slides Advantage of Supercomputers Europe s position The Scientific Case / Extrapolations from Germany Energy Consumption 54

56 6 years technolgy advantage with a #1 system Top 500 list 11/07 6 years after 12 years the power is on the desk 55

57 Europe s current position in HPC PFlops Aggregated LINPACK Performance in PetaFlop/s in November Top 500 Lists 91% of European HPC-power is within PRACE countries 56

58 HET: The Scientific Case Weather, Climatology, Earth Science degree of warming, scenarios for our future climate. understand and predict ocean properties and variations weather and flood events Astrophysics, Elementary particle physics, Plasma physics systems, structures which span a large range of different length and time scales quantum field theories like QCD, ITER Material Science, Chemistry, Nanoscience understanding complex materials, complex chemistry, nanoscience the determination of electronic and transport properties Life Science system biology, chromatin dynamics, large scale protein dynamics, protein association and aggregation, supramolecular systems, medicine Engineering complex helicopter simulation, biomedical flows, gas turbines and internal combustion engines, forest fires, green aircraft, virtual power plant 57

59 Status and Requirements, e.g.: German Case Scientific Field (numbers in TeraFlop/s) Climate and Earth System Research Geophysics Nanostructure Physics Solid-State Physics Computational Fluid Dynamics Astrophysics Elementary Particle Physics and Physics of Hadrons and Nuclei Materials Science Theoretical Chemistry Soft Matter Biophysics and Bioinformatics Plasma Physics A. Bode, W. Hillebrandt, and Th. Lippert: German Scientific Case for the BMBF, 8/ >500 >1000 >200 >1000 >1000 >500 >1000 >500 >300 >200 >1000 >500 58

60 Exascale Data Centre Challenges LBNL IJHPCA Study for ~1/5 Exaflop for Climate Science in 2008 Extrapolation of Blue Gene and AMD design trends Estimate: 20 MW for BG and 179 MW for AMD DOE E3 Report Extrapolation of existing design trends to Exascale in 2016 Estimate: 130 MW DARPA Study More detailed assessment of component technologies Estimate: 20 MW just for memory alone, 60 MW aggregate extrapolated from current design trends The current approach is not sustainable! More holistic approach is needed! 59

61 Power Bill of German Data Centers PRACE EC Review WP8 March 5-6, Brussels 60

Outline. PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich ORAP, Lille, March 26, 2009

Outline. PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich ORAP, Lille, March 26, 2009 PRACE A Mid-Term Update Dietmar Erwin, Forschungszentrum Jülich ORAP, Lille, March 26, 2009 Outline What is PRACE Where we stand What comes next Questions 2 Outline What is PRACE Where of we stand What