Consorzio COMETA FESR

Transcription

1 Consorzio COMETA FESR Visualization Element: towards the definition of a new Grid service Giuseppe ANDRONICO (1), Roberto BARBERA (1)(2), Andrea FORNAIA (1), Marcello IACONO MANNO (3) and Giuseppe LA ROCCA (1) (1) INFN Na*onal Ins*tute of Nuclear Physics, Division of Catania, Italy (2) Department of Physics and Astronomy of the University of Catania, Italy (3) Consorzio COMETA, Catania, Italy ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan

2 Outline The Consorzio COMETA and the Sicilian Grid infrastructure; Introduction to the Visualization Element service Motivations; Design and Implementation; Architecture; The Stereoscopy; Examples; Blender; VISIVO; Future plans. Summary and Conclusions; References. ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 2

3 The Italian e-infrastructure (interoperable with common communication protocols) The Network FESR The Grid ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 3

4 The COMETA Consortium ( ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 4

5 Objectives of an e-infrastructure in Sicily Create a Virtual Laboratory in Sicily, both for scientific and industrial applications, built on top of a grid infrastructure Connect the Sicilian e-infrastructure to those already existing in Italy, in Europe, and in the rest of the world improving the scientific collaboration and increasing the competitiveness of e-science and e-industry made in Sicily Disseminate the grid paradigm through the organization of dedicated events and training courses Trigger/foster the creation of spin-offs in the ICT area in order to reduce the brain drain of brilliant young people to other parts of Italy and beyond ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 5

6 The COMETA Infrastructure ~2500 CPUs ~250 TB Catania ~15 M in 3 years >350 people involved! ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 6

7 Computing and storage resources available 1. ~2000 cores AMD Opteron GB of RAM per core 3. LSF as LRMS everywhere 4. Infiniband- 4X at all sites TB of raw disk storage 2. Distributed/parallel GPFS filesystem ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 7

8 Commercial software available ( ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 8

9 Motivations Several Grid applications (e.g.: FLUENT) running in the COMETA Infrastructure produce media content (e.g.: images) that can be collected and encoded as a unique video to be visualized also when the job that produces the images is still running. This feature maybe helpful for both demonstration purposes or to control job evolution. Visualiza*on Element is a post-processing tool allowing to show, starting from a list of images sequence, the video output produced by a producer application. 64 cores for 14 CPU days Environment protection Fig.1 - Some frames produced by FLUENT and streamed by the VisualElement service. ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 9

10 Deploying the VisualElement on Grid VisualElement uses mencoder, VLC VideoLAN Client, Xva and FFMPEG software packages to encode the media content and produce a video that will be send in streaming. The needed packages for Visualization Element can be downloaded from repos. Top BDII Published a new software tag, VO-cometa-VLC-0.9.9a, for the service. A JDL file is created on-the-fly using a Grid portal to call the service with the corresponding inputs. A first prototype of the Visualization Element on-line streaming tool has been deployed for the COMETA Grid infrastructure on board of the GENIUS Portal powered by EnginFrame from NICE srl ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 10

11 Integration of VisualElement in COMETA Two different implementations of the service are available as shown in fig. 1 and fig. 2 Based on glite 3.2 CREAM CE Based on LCG Grid CE Fig. 1 Genera*on of the video streaming with a single job Fig. 2 Genera*on of the video output using a DAG ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 11

12 Commercial & industrial HPC applications on Grid: CFD simulations with Fluent 64 cores for 14 CPU days Environment protec*on ( Le Marmore falls) 48 cores for 7 CPU days Engine design ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 12

13 What s next? The Stereoscopy It is the ability of creating, or enhancing, the illusion of depth in a flat image: by presenting two offset images, respectively for left and right eye, combined together in a single one, using some optical technique; This is exactly what our brain does with the images taken from the eyes. To implement the Stereoscopy, we can use three different methods: Using two separated sources: the user wears eyeglasses to combine the images taken from them; Using only one source: the user wears eyeglasses to split the image taken from it; Using a self-stereoscopic source: that splits the images directing the light into the viewer's eye, with no need of eyeglasses. ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 13

14 Stereoscopic streams in practice We had to produce for each eye, two separated streams of images observing the simulation from two different point of view; Then, we have to combine them in a single stream using appropriated video drivers, in according to the chosen stereoscopic technique; e.g., the anaglyph one. ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 14

15 First use case The service has been tested for the first time encoding a BLENDER application. The The application reconstructed video will be send in streaming via UDP (default port 1234), generates to the client s for each IP using frame VLC. 2 different images; ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 15

16 Second use case VIsIVO Server for COMETA Grid can create a 3D view from data tables; It runs on every Linux based system: The main characteristic is that VisIVO Server gives support for huge data; No limits (in principle) are fixed to visualize your data. Some limits exist up to now, but the next version will solve these problems; The FITS and HDF5 data format will be included among supported formats; Output data from Gadget code will be soon supported; The project is completed since October ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 16

17 Any questions, comments or remarks are very welcome. Please contact: giuseppe. ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 17

18 References & Links G. Andronico, R. Barbera, M. Iacono Manno and G. La Rocca. VisualGrid: On-Line Video Streaming for Application Control and Demonstration. Proceedings of the Final Workshop of Grid Projects PON Ricerca , Avviso 1575, 2009 ISGC 2011 & OGF March 2011, Academia Sinica, Taipei, Taiwan 18