First Experience with PCP in the PRACE Project: PCP at any cost? F. Berberich, Forschungszentrum Jülich, May 8, 2012, IHK Düsseldorf

First Experience with PCP in the PRACE Project: PCP at any cost? F. Berberich, Forschungszentrum Jülich, May 8, 2012, IHK Düsseldorf

Overview WHY SIMULATION SCIENCE WHAT IS PRACE PCP IN THE VIEW OF A PROJECT ANALYSE OF PCP EXAMPLE: WHOLE SYSTEM DESIGN FOR ENERGY EFFICIENT HPC

WHY SIMULATION SCIENCE

Simulation: the third pillar of science Traditional Scientific - Technical Paradigm 1. Create a theoretical model 2. Perform experiments and /or construct a prototype Galileo Galilei folio 116v (before 1609) 4

Simulation: the third pillar of science 5

Simulation: the third pillar of science Constraints Too difficult (construct bigger wind tunnel) Too expensive (construct one-way aircraft) Too slow (wait for the climate change) Too dangerous (drugs, climate, ) Computational Science Paradigm 3. Use of HPC resources for simulation of the phenomenon based on known physics laws and efficient numerical methods

Why do we need PRACE? Money: No one system fits all problems Quality: Scientific Excellence: European Peer Review Tier-0: HPCsystems of the highest performance class Access: guarantee independent access for all computational scientists and European industry Sustainability: EU and national governments are motivated through PRACE to establish robust and persistent funding schemes 7

WHAT IS PRACE

PRACE History An Ongoing Success Story HPC part of the ESFRI Roadmap; creation of a vision involving 15 European countries Creation of the Scientific Case Signature of the MoU Creation of the PRACE Research Infrastructure HPCEUR HET PRACE Initiative PRACE RI 2004 2005 2006 2007 2008 2009 2010 2011 2012 PRACE Preparatory Phase Project PRACE-1IP PRACE-2IP PRACE-3IP 9

Realizing the ESFRI Vision for a HPC RI European HPC-facilities at the top of an HPC provisioning pyramid Tier-0: European Centres for Petaflop/s Tier-1: National Centres Tier-2: Regional/University Centres Creation of a European HPC ecosystem HPC service providers on all tiers Scientific and industrial user communities The European HPC hard- and software industry Other e-infrastructures Tier-0 European centres Tier-1 National centres Tier-2 Regional/University centres # of systems capabili ity

April, 23 rd 2010 creation of the legal entity (AISBL) PRACE with seat location in Brussels, Belgium 24 PRACE Members 68+ Million from EC FP7 for preparatory and implementation Grants INFSO-RI-211528, 261557, and 283493 Complemented by ~ 50 Million from PRACE members Interest by Denmark, Israel,

Three pillars of the PRACE Mission HPC for Industry HPC for Science HPC by Vendors Guaranteeing in Europe the independent access to HPCcompetence for industry Implement the ESFRI vision of a European HPC-service at the top of an HPC provisioning pyramid Helping European (hard- & software) vendors to foster their technology and HPCcompetence

Tier-0 Tier-1 Tier-2 PRACE is building the top of the pyramid... First production system available: 1 Petaflop/s IBM BlueGene/P (JUGENE) at GCS (Gauss Centre for Supercomputing) partner FZJ (Forschungszentrum Jülich) Italy and Spain expect to deploy their own Tier-0 systems from 2012. Second production system available: Bull Bullx CURIE at GENCI partner CEA. Full capacity of 1.8 Petaflop/s reached by late 2011. Fourth production system available by mid 2012: 3 Petaflop/s IBM (SuperMUC) at GCS partner LRZ (Leibniz-Rechenzentrum). Third production system available by the end of 2011: 1 Petaflop/s Cray (HERMIT) at GCS partner HLRS (High Performance Computing Center Stuttgart). Upgrade to 4-5 Petaflop/s planned in 2013.

PCP IN THE VIEW OF A PROJECT

3 nd Implementation Project Funding up to 19 Million (PCP budget 25% - 50%) Under Preparation: FZJ/JSC/PRACE PMO Mid 2012 Mid 2014 Planned: Pre-commercial procurement exercise Planned: Industry application focus 15

Analysis Pre-commercial procurement R&D services only: R&D activities >50 % of the overall budget Application of risk-benefit sharing: shared IPR. The public purchase does not reserve the R&D results exclusively for its own. A competitive procurement designed to exclude state aid: PCP R&D work has to be done by marked prices. 16

Foster Innovation in Europe through PCP Imbalance between European and US economies benefit from basic research funding Basic and applied R&D similar (2% GDP in EU v 2.5% in US) But public procurement of innovative R&D results prior to full commercialisation is not ( 2.5 billion in EU v 50 billion in US) PCP is a procurement model designed to fill this gap Basic idea is (c.f. DARPA s HPCS programme) Public procurers join together to run a pre-commercial procurement process Phased process where several suppliers initially respond Final results delivered as pre-commercial products or services to procurers but IPR remains with suppliers 17

Foster Innovation in Europe through PCP Location of R&D and operational activities related to the PCP: Relevant portion of R&D activities located in the European Economic Area or in a country having concluded a Stabilisation and Association Agreement Main players in HPC are mainly based in US, only few companies in EU EU: Mainly Integrators, Software and Network Competences Need to start to developing and supporting HPC industry in EU 18

Preconditions for PCP Market PCP allows to restrict the choice of companies, e.g. location in EU (relevant portion), place of performance critical amount of suited companies needed Topic no co development possible conflict of interest contracting public body is not able to do this R&D or consider the R&D topic in question as not worth for in house research R&D main objective of the procurement is R&D no procurement of goods / systems but prototypes 19

PRACE PCP Pilot versus Preconditions Market European HPC vendors are rare US dominated field, but also China and Japan with increasing success (TOP 500) Topic PRACE partners are the EU competence for HPC PRACE partners developing with Intel / IBM new technologies for the exascale systems (ExaScale Lab) R&D main objective of the procurement is R&D no procurement of goods / systems but prototypes, codes 20

PCP The Process 21

Identification of the key technology to be addressed by PCP Preparation of the procurement process and the procurement documents Procurement process Each contract covers the three phases but only phase 1 is committed, phase 2 and 3 are options with a detailed content to be defined later PCP Phase 1 PCP Phase 1 PCP Phase 1 PCP Phase 1 Outcome: a set of theoretical solutions to investigate further Outcome: a solution validated at prototype level and ready to be tested at the level of pilot (=preproduction) system(s) PCP Phase 2 PCP Phase 2 PCP Phase 2 PRACE PCP Timeline PCP Phase 3 PCP Phase 3 Solution exploration Validation of solutions with Prototypes Construction of Pilot systems Evaluation of the pilot system at preproduction level M6 M12 M21 M30 M30 M48 22

Joint PCP Procurement EPCC (UK) JUELICH (DE) PRACE AISBL CINECA (IT) GENCI (FR) Final set of PCP consortium not yet defined Difficult Negotiations with funding agencies due to new PCP process Contracting body: PRACE AISBL as a public body Each partner has a different tax status e.g. JUELICH full tax payer but entitled to deduct input tax GENCI full tax payer CINECA limited tax payer PRACE AISBL - non-taxable person 23

Future System Architecture Projections/Targets Systems 2010 2012 2015 2018 Difference Today & 2018 System Peak [PF] 2 25 200 1000 O(1000) Power [MW] 6 6-20 15-50 20-80 O(10) System Memory [PB] 0.3 0.3-0.5 5 32-64 O(100) GB RAM/Core 0.5-4 0.5-2 0.2-1 0.1-0.5 -O(10) Node Performance [GF] 125 160-1000 500-7000 1000-10000 O(10)-O(100) Cores/Node 12 16-32 100-1000 1000-10000 O(100)-O(1000) Node memory BW [GB/s] 40 70 100-1000 400-4000 O(100) Number of nodes ~20.000 10.000-100.000 5000-50.000 100.000-1.000.000 O(10)-O(50) Total concurrency ~240.000 O(10 6 ) O(10 7 ) O(10 9 ) O(10.000) MTTI days days O(1 day) O(1 day) -O(10) Source: Rick Stevens and Andy White, IESP Meeting, Oxford 2010

Whole System Design for Energy Efficient HPC Transformation of HPC Data Centre efficiency through the production of energy efficient hardware and systemware components Development of extreme cooling technologies in order to remove more than 90% of the heat from the system Demonstration of an integrated whole system design for low total energy-to-solution values as enabled by highly energy efficient computing and extreme cooling technologies 25

Market survey PCP approach in HPC in Europe is new exchange with all potentially interested suppliers on both the technical challenge of energy efficiency and the practical implementation of a PCP in HPC Workshop on the usage of Pre- Commercial Procurement (PCP) in HPC 26

Conclusions PCP is a powerful tool to foster innovation in Europe PCP should be used by Funding Agencies and Ministries, Regions, Cities FP8 Horizon 2020: PCP without endanger the excellence principle Conflict of interest: own competence co development PRACE will pilot PCP in HPC Find right topics and right partners Open dialogue 27

30.8.2011 Thomas Lippert - FZJ 28

Questions? Forschungszentrum Jülich GmbH Dr. Florian Berberich Leo-Brandt-Str. 52425 Jülich Germany E-mail: f.berberich@fz-juelich.de 29