Einsatz von UNICORE in Rechenzentren

Transcription

1 Mitglied der Helmholtz-Gemeinschaft Einsatz von UNICORE in Rechenzentren Björn Hagemeier

2 Part: About Us Björn Hagemeier Folie 2

3 Forschungszentrum Jülich and JSC Björn Hagemeier Folie 3

4 Forschungszentrum Jülich and JSC Björn Hagemeier Folie 3

5 JUQUEEN IBM Blue Gene/Q 28 racks, 458,752 cores PowerPC A2 a.6ghz 16 cores per node 5.8 Petaflop/s peak 460 TByte main memory 5D network Björn Hagemeier Folie 4

6 JURECA 1872 compute nodes Intel Haswell with 2x12 GHz 75 compute nodes equipped with 2 NVIDIA K80 GPUs DDR4 memory (2133MHz) 1605 nodes with 128GiB memory 128 nodes with 256 GiB memory 64 nodes with 512 GiB memory 12 visualization nodes 2 NVidia K40 per nodes 10 nodes with 512 GiB memory 2 nodes with 1024 GiB memory Total of 45,216 cores 100 GiB/s storage connection Björn Hagemeier Folie 5

7 JUST Juelich Storage Cluster IBM GPFS 20.3 PB online storage 220 GB/s Fileserver for HPC-Systems: JUQUEEN, JURECA DEEP (Dynamical Exascale Entry Platform) Björn Hagemeier Folie 6

8 Tape Libraries Actual capacity: 99 PB Theoretical capacity: 141 PB (16600x8.5TB) Tape drives: 48 Libraries: Björn Hagemeier Folie 7

9 Part: UNICORE Björn Hagemeier Folie 8

10 UNICORE As We See It Today A federation software suite Secure and seamless access to compute and data resources Focus on scientific applications and workflows Complies with typical HPC centre policies Complete solutions: APIs, clients, services,... Java/Python based, supports UNIX, MacOS, Windows and many resource management systems (Torque, Slurm, SGE,...) Long development history (since 1997) Open source, BSD licensed, visit Björn Hagemeier Folie 9

11 Concepts UNICORE UNIform Access to COmputing REsources Site: a resource such as an HPC system including storage Job: submitted through JSDL including data staging, resource requirements, executable definition and parameters Hadoop (Yarn) jobs possible, too, in conjunction with HDFS Resources: features of sites in terms of capacity and capability Storages: a view into file systems at a certain base directory (mount point). Can be storage external to the site, e. g. Swift, S3, HDFS, CDMI, XtreemFS Applications: abstractions of applications hiding site-local specificities, e. g. installation paths or module activations Workflows: a series of job executions guided by control structures, i. e. visual programming Björn Hagemeier Folie 10

12 Architecture Björn Hagemeier Folie 11

13 Unicore Main Services Compute TargetSystemFactory TargetSystem JobManagement Reservations Storage and Data StorageFactory StorageManagement FileTransfer Metadata Workflow Workflowenactment Task Execution Resource Broker Registry Björn Hagemeier Folie 12

14 Default Setup Access to resource manager and file system via TargetSystemInterface (TSI) daemon installed on the cluster login node(s) Björn Hagemeier Folie 13

15 Job Execution Björn Hagemeier Folie 14

16 Storage Access The UNICORE Storage Management Service ( SMS ) provides a filesystem-like view of data Typical functions mkdir, delete, ls, chmod etc Start tile transfers Import/export of data from/to the user s local machine Send/receive of data from other servers Various supported file transfer protocols Björn Hagemeier Folie 15

17 File Transfers Björn Hagemeier Folie 16

18 Metadata Management Service MMS Automatic extraction Manual editing of metadata Searching Björn Hagemeier Folie 17

19 Applications General Identified by name and version Site specifics Pre and post commands for environment setup and tear down Acquire and return licenses MPI Support for application metadata Björn Hagemeier Folie 18

20 Generic Applications Automated generation of GUIs UNICORE Rich Client and Portal support application metadata Example <jsdl:argument Description="Check input file" Type="boolean" Default="..." ValidValues="true false" DependsOn="..." Excludes="..." IsEnabled="false" IsMandatory="false">+v$CHECK?</jsdl:Argument> Possible types: string, boolean, int, double, filename, choice. Used to be defined by site administrators Björn Hagemeier Folie 19

21 User-Defined Applications Allow mixing system and user defined applications Encourage users to play with and develop their own application definitions Repository of common application definitions Realized by merging system and user specific IDB contributions Users cannot change a site s resources and thus not go beyond administrator limits Björn Hagemeier Folie 20

22 Workflow Features Simple graphs (DAGs) Workflow variables Loops and control constructs while, for-each, if-else Conditions Exit code, file existence, file size, workflow variables Clients UNICORE Rich client Commandline client Björn Hagemeier Folie 21

23 Authentication and Authorization AAI, for short In addition to its own, home-grown user management solution, aka. XUUDB, UNICORE supports SAML-based authentication. PULL and PUSH-mode are possible Typically only need a few attributes role (user, server, admin), xlogins, groups Björn Hagemeier Folie 22

24 UNITY IdM Identity Relationship Management Complete solution for identity, federation and inter-federation management Can serve as SP and IdP at the same time. Use SAML 2, OAuth 2, OIDC, LDAP as upstream IdPs Serve as IdP for SAML 2 (Web SSO, SOAP, PAOS bindings), SAML 2 Web & SOAP UNICORE Profile, OIDC, OAuth Björn Hagemeier Folie 23

25 UNITY IdM Infrastructures UNICORE DFN AAI for authentication Still need proper account at JSC Björn Hagemeier Folie 24

26 UNITY IdM Infrastructures Björn Hagemeier Folie 25

27 Part: Installation Björn Hagemeier Folie 26

28 Installation General Latest releases of most important components linked on main website Detailed download section at contains all components Packages are hosted on SourceForge Björn Hagemeier Folie 27

29 Installation Basic Core Server Bundle Servers/Core/ Content Gateway UNICORE/X Registry TSI XUUDB Requirements OpenJDK 8 or Oracle Java 8 Python 2.7 or 3.x for the TSI Björn Hagemeier Folie 28

30 Installation Workflow Servers/Workflow/ Content Workflow Engine Resource broker aka. Service Orchestrator Björn Hagemeier Folie 29

31 Installation Federation Common Registry All services need to publish their availability to a common registry Can publish to multiple registries Clients support multiple registries Authentication Individual registrations (certificate) Identity federation, e. g. via UNITY Björn Hagemeier Folie 30

32 Acknowledgements Most slides shamelessly copied from my colleague Bernd Schuller. Other team members Valentina Huber, André Giesler, Maria Petrova-El Sayed, Jedrzej Rybicki, Rajveer Saini and many others at JSC Krzysztof Benedyczak, Marcelina Borcz, Rafa l Kluszczynski, Piotr Ba la and others at ICM / Warsaw University Richard Grunzke and others at Technical University Dresden Students: Burak Bengi, Maciej Golik, Konstantine Muradov... many others who reported bugs, suggested features, contributed code and provided patches Björn Hagemeier Folie 31