INCITE Program Overview May 15, Jack Wells Director of Science Oak Ridge Leadership Computing Facility

Transcription

1 INCITE Program Overview May 15, 2012 Jack Wells Director of Science Oak Ridge Leadership Computing Facility

2 INCITE: Innovative and Novel Computational Impact on Theory and Experiment INCITE promotes transformational advances in science and technology through large allocations of computer time, supporting resources, and data storage at the Argonne and Oak Ridge Leadership Computing Facilities (LCFs) for computationally intensive, large-scale research projects. 2

3 Provide access to LCFs: More than 2.7 billion core hours awarded in % Director s Discretionary 270 million core-hours 30% ALCC Up to 810 million core-hours 60% INCITE 1.7 billion core-hours 3

4 Allocation Programs at the LCFs 60% INCITE 30% ALCC 10% Director s Discretionary Mission High-risk, high-payoff science that requires LCF-scale resources* High-risk, high-payoff science aligned with DOE mission Strategic LCF goals Call 1x/year (Closes June) 1x/year (Closes February) Rolling Duration 1-3 years, yearly renewal 1 year 3m,6m,1 year Typical Size projects 10M - 100M corehours/yr projects 1M 75M core-hours/yr. 100s of projects 10K 1M corehours Review Process Scientific Peer-Review Computational Readiness Scientific Peer-Review Computational Readiness Strategic impact and feasibility Managed By INCITE management committee (ALCF & OLCF) DOE Office of Science LCF management Availability Open to all scientific researchers and organizations Capability >20% of cores 4

5 Is INCITE right for me? INCITE seeks computationally intensive, large-scale research projects with the potential to significantly advance key areas in science and engineering. 1 Impact criterion High-impact science and engineering 2 Computational leadership criterion Computationally intensive runs that cannot be done anywhere else 3 Eligibility criterion INCITE grants allocations regardless of funding source (ex. DOE, NSF, private, etc) Non-US-based researchers are welcome to apply Call for 2013 proposals opens in Spring 2012 The INCITE program seeks proposals for highimpact science and technology research challenges that require the power of the leadershipclass systems 5

6 Twofold review process Peer review: INCITE panels Computational readiness review: LCF centers New proposal assessment Scientific and/or technical merit Appropriateness of proposal method, milestones given Team qualifications Reasonableness of requested resources Technical readiness Appropriateness for requested resources Change in scope Renewal assessment Met milestones On track to meet future milestones Scientific and/or technical merit Met technical/ computational milestones On track to meet future milestones Award Decisions INCITE Awards Committee comprised of LCF directors, INCITE program manager, LCF directors of science, sr. management 6

7 2012 INCITE allocations 28 new projects, 32 renewals Acceptance rate: 33% for new proposals, 91% for renewals Distribution of INCITE time by science domain Awarded 1.67 billion hours for CY

8 2012 INCITE award demographics PI Affiliations 2012 INCITE Awards PI Demographics 2012 INCITE Awards 8

9 Diversity of INCITE science Simulating a flow of healthy (red) and diseased (blue) blood cells with a Dissipative Particle Dynamics method. - George Karniadakis, Brown University Demonstration of high-fidelity capture of airfoil boundary layer, an example of how this modeling capability can transform product development. - Umesh Paliath, GE Global Research Provide new insights into the dynamics of turbulent combustion processes in internal-combustion engines. -Jacqueline Chen and Joseph Oefelein, Sandia National Laboratories Calculating an improved probabilistic seismic hazard forecast for California. - Thomas Jordan, University of Southern California Modeling charge carriers in metals and semiconductors to understand the nature of these ubiquitous electronic devices. - Richard Needs, University of Cambridge, UK Other INCITE research topics Glimpse into dark matter Supernovae ignition Protein structure Creation of biofuels Replicating enzyme functions Global climate Regional earthquakes Carbon sequestration Turbulent flow Propulsor systems High-fidelity simulation of complex suspension flow for practical rheometry. - William George, National Institute of Standards and Technology Membrane channels Protein folding Chemical catalyst design Combustion Algorithm development Nano-devices Batteries Solar cells Reactor design Nuclear structure 9

10 Scaling of Codes on 224,256 core Jaguar Science Area Code Contact Institution Cores Total Performance Notes CFD MoBo Rahimian Georgia Tech 200, TF 2010 Gordon Bell Winner Seismology AWP-ODC Cui SDSC 223, TF 2010 Gordon Bell Finalist Materials DRC Kozhevnikov ETH Zurich 186, PF Geoscience P4est Burstedde UT-Austin 224, Gordon Bell Honorable Mention 2010 Gordon Bell Finalist Materials DCA++ Schulthess ETH & ORNL 213, PF* 2008 Gordon Bell Winner Materials WL-LSMS Eisenbach ORNL 223, PF 2009 Gordon Bell Winner Chemistry NWChem Apra ORNL PNNL 224, PF 2009 Gordon Bell Finalist Nano Materials OMEN Klimeck Purdue 221, PF 2011 Gordon Bell Finalist Seismology SPECFEM3D Carrington SCEC 149, TF 2008 Gordon Bell Finalist Weather WRF Michalakes NREL 150, TF Combustion S3D Chen Sandia 144, TF Fusion GTC Chang PPPL 102, billion Particles / sec Materials LS3DF Lin-Wang Wang Berkeley 147, TF 2008 Gordon Bell Winner Chemistry MADNESS Harrison ORNL UTenn 140, TF 10

11 INCITE resources to request OLCF The Cray XK6 is in production in CY 2012 with 960 Fermi GPUs installed as Titandev The Cray XK6 (with Kepler GPU) will be in production in CY 2013 with at least 14,592 Kepler GPUs Projects may apply for production time on Titan in CY

12 Key questions to ask yourself Is both the scale of the runs and the time demands of the problem of LCF scale? Yes, I can t get the amount of time I need anywhere else. Yes, I my simulations are too large to run on other systems. Do you need specific LCF hardware? Yes, the memory and I/O available here are necessary for my work. Do answer these questions in the proposal. This is especially helpful for the computational readiness reviewers. 12

13 Our ensemble policy or, Can I meet the computationally intensive criterion by loosely coupling my jobs? Possibly yes, If you require large numbers of discrete or loosely coupled simulations where time-to-solution is an untenable pacing issue, and If a software workflow solution (e.g., pre- and post-processing scripts that automate run management and analysis) is provided to facilitate this volume of work. Probably no, If by decoupling the simulations the work could be effectively carried out on a smaller resource within a reasonable time-to-solution. 13

14 Request a start up account Director s Discretionary Proposals considered year-round Award up to millions of hours Allocated by LCF center directors Director s Discretionary (DD) requests can be submitted anytime DD may be used for porting, tuning, scaling in preparation for an INCITE submittal Submit applications at least 2 months before INCITE Call for Proposals closes Oak Ridge DD Program: 14

15 2013 INCITE award announcements Notice comes from INCITE Manager, Julia White, in November 2012 Welcome and startup information comes from centers Agreements to sign: Start this process as soon as possible! Getting started materials: Work closely with the center Centers provide expert-to-expert assistance to help you get the most from your allocation Scientific Liaisons and Catalysts (OLCF / ALCF) 15

16 Relevant links INCITE Program Oak Ridge Discretionary Program Contact the center if you d like to request Discretionary time for benchmarking 16

17 Contacts For details about the INCITE program: For details about the centers: